[ibrahimnd2000]

route_orchestrated_disaggregated_request sends the prefill request with max_tokens=1 and the client's original body to decode. Two cases break the "one token on prefill" assumption.

1. min_tokens > 1 fails the whole request (HTTP 400). The client's min_tokens is copied into the prefill request, and vLLM rejects min_tokens > max_tokens:

Prefill failed: {"error":{"message":"min_tokens must be less than or equal to max_tokens=1, got 200.","type":"BadRequestError","code":400}}

This happens with or without a client max_tokens. send_request_to_prefiller (the other P/D mode) already drops min_tokens; this does the same in the orchestrated path. The decode request keeps it.

2. /v1/responses returns garbage. vLLM's Responses API ignores max_tokens: it reads max_output_tokens, and when that is unset the limit is the rest of the context. So the prefill generates a whole answer and returns kv_transfer_params for prompt + output, and decode builds on KV that does not match its own prompt.

Measured on vLLM 0.31 with the NIXL connector, one 162-token prompt, posting the router's prefill body directly to a prefill pod:

Prefill request	Output tokens	remote_num_tokens
/v1/chat/completions	1	162
/v1/responses (today)	40	204
/v1/responses + max_output_tokens: 1	1	162

End to end through the router, /v1/responses returned unreadable text from the first token. With this change it returns the expected reasoning and tool call, the same as chat completions. The prefill request now also gets max_output_tokens=1 on /v1/responses, or whenever the body already has max_output_tokens. The decode request keeps the client's value.

Tests: src/tests/test_disaggregated_orchestrated_prefill_limits.py runs the orchestrated flow with a fake client and checks three things:

• on chat with min_tokens: 200, with and without max_tokens, the prefill request has no min_tokens and max_tokens 1, while decode keeps both client values;
• on /v1/responses, with max_output_tokens 300 or unset, the prefill request gets max_output_tokens 1, while decode keeps the client's value;
• chat gets no max_output_tokens.

4 of the 5 cases fail without the fix. The rest of src/tests passes (265). test_multipart_proxy.py couldn't be collected locally because httpx isn't installed. pre-commit (black, isort, ruff, codespell) passes.

We run both fixes in production on a GLM-5.3 P/D deployment (vLLM 0.31, NIXL over EFA): min_tokens requests and /v1/responses both return correct output.

Related orchestrated-P/D fixes: #1117, #1122, #1123.

[gemini-code-assist]

Code Review

This pull request addresses issues in the orchestrated prefill/decode flow where prefill requests could fail or generate more than one token. It removes min_tokens from the prefill request to prevent HTTP 400 errors and sets max_output_tokens to 1 for the Responses API. A new test suite is also added to verify these changes. The reviewer suggested also popping min_completion_tokens from the prefill request to avoid similar validation failures when clients use that parameter.

[gemini-code-assist]

In addition to min_tokens, the OpenAI API and vLLM also support min_completion_tokens. If a client sends min_completion_tokens with a value greater than 1, it will cause the same BadRequestError (HTTP 400) during the prefill request because max_tokens is set to 1. We should also pop min_completion_tokens from the prefill request to prevent this failure.

Suggested change

	# Avoid min_tokens > max_tokens=1 conflict in vLLM SamplingParams (400);
	# the decode request keeps the client's min_tokens.
	prefill_request_json.pop("min_tokens", None)
	# Avoid min_tokens / min_completion_tokens > max_tokens=1 conflict in vLLM SamplingParams (400);
	# the decode request keeps the client's min_tokens.
	prefill_request_json.pop("min_tokens", None)
	prefill_request_json.pop("min_completion_tokens", None)

[ibrahimnd2000]

Thanks. I checked this against vLLM 0.31: min_completion_tokens isn't a vLLM request field (no reference under vllm/entrypoints; the chat request has min_tokens only), and it isn't in the OpenAI API either. A prefill-style request with max_tokens: 1, min_completion_tokens: 200 returns 200 with 1 completion token, since the unknown field is ignored. So it can't cause the 400, and I've left it out to keep the prefill body otherwise unchanged. Happy to add it if a backend that honours it shows up.

[ptimizeroracle]

ran this locally at 3d9aef8, not a maintainer, checking the claim

• src/tests/test_disaggregated_orchestrated_prefill_limits.py: 5 pass with the change
• with request.py put back to the merge base, 4 of them fail: both test_min_tokens_only_on_the_decode_request cases and both test_responses_prefill_generates_one_token cases
• send_request_to_prefiller does already pop min_tokens (the comment there says the same thing about max_tokens=1), so this brings the orchestrated path in line with the other p/d mode
• full src/tests run on the branch: 268 passed, 1 failed, the same test_prefixaware_router test that fails on main on my machine
• merges clean with #1132 on the same route and the combined tests pass

what i could not check: the vllm side of the description (the 400 on min_tokens > max_tokens, and /v1/responses ignoring max_tokens). vllm does not install on my machine, so that part is taken from the description

the router change looks correct to me

[ibrahimnd2000]

Thanks for verifying! For the vLLM side, from our deployment (vLLM 0.31.0):

• min_tokens > max_tokens: the prefill request comes back as a 400, min_tokens must be less than or equal to max_tokens=1, so any client sending min_tokens > 1 fails in orchestrated P/D.
• /v1/responses: the Responses API caps output with max_output_tokens and ignores max_tokens. On a 162-token prompt the prefill pod generated 40 tokens and returned remote_num_tokens: 204, so decode built on KV that didn't match its prompt and produced gibberish from the first token. With this change the prefill returns remote_num_tokens: 162 and the answer is correct.

The k8s-discovery-e2e-test failure here is the CI cluster's vLLM pods not becoming ready within 600 s (startup probe refused); the same job also fails on unrelated PRs this week.