Add step tests for chunked prefill

[maxdebayser]

Description

This PR add step by step execution tests for the chunked prefill scheduler. It tests several scenarios:

1. Single sequence with several chunks
2. Two sequences scheduled at the same time, one with 1 chunk and the other with 4
3. Same as 2 but with interleaving disabled to test this debugging feature
4. Same as 2 but the second requests arrives later.

The tests show that the interleaving of requests is working as expected.

[github-actions]

👋 Hi! Thank you for contributing to vLLM support on Spyre.
Just a reminder: Make sure that your code passes all the linting checks, otherwise your PR won't be able to be merged. To do so, first install the linting requirements, then run format.sh and commit the changes. This can be done with uv directly:

uv sync --frozen --group lint --active --inexact

Or this can be done with pip:

uv pip compile --group lint > requirements-lint.txt
pip install -r requirements-lint.txt
bash format.sh

Now you are good to go 🚀

[maxdebayser]

The test failure seems related to state carried over from tests like test_prefill_tkv_too_big. If run together with this test, test_cp_prefill_interleave1 fails locally as well.

[maxdebayser]

bot:test

[wallashss]

LGTM

[maxdebayser]

@tjohnson31415 , the 514 model len here is what would trigger the hang before your fix.

[tjohnson31415]

Tests that modify VLLM_DT_MAX_BATCH_TKV_LIMIT should be using monkeypatch which would automatically restore the value after the test. Can we use that instead of having a special case reset here?

[maxdebayser]

It's bit more involved than a monkey patch because we have to replicate some logic from platform.py. But it's possible to do so with a fixture if you prefer.

[tjohnson31415]

I don't mean to set the actual value of VLLM_DT_MAX_BATCH_TKV_LIMIT with monkeypatch but using setenv will register to monkeypatch that it needs to restore the previous value even if something in the test overrides it.

I thought we had an example of this in the code, but I can't find it now.

[maxdebayser]

The problem is the interaction with the model cache. The setting/resetting of this variable has to be tied to the life cycle of the llm engine but pytest is not aware of that since it's not a fixture. Do you remember if the example you're referring to also dealt with this problem?

[tjohnson31415]

Hmm, I don't recall the previous case dealing with the model cache. I'm not very familiar with how the caching works...

[tjohnson31415]

LGTM