refactor(responses): make openai_stream_events IRR-only - #1013
Conversation
praxis-bot
left a comment
There was a problem hiding this comment.
praxis-bot review: refactor(responses): make openai_stream_events IRR-only
Clean refactor that hardens the contract: openai_stream_events now always composes logical streams and must live inside an iterative_request_router step. The logical_stream config flag is removed, the fail-closed guard rejects misconfigured placements, and all examples/tests/docs are updated accordingly. The architectural invariant is well-enforced and the test coverage (both unit and functional integration) is thorough.
One medium finding below.
Reviewed files: 23/23
Classification: Medium (1)
praxis-bot review on praxis-proxy#1013 flagged that the IterationState placement guard's accept branch (arm inside an iterative_request_router step) had no unit coverage — only the reject branch did. IterationState cannot be constructed outside praxis-filter (its fields are pub(crate) with no public or test constructor), so the suggested test-only constructor is unavailable without an upstream change. Extract the guard's fail-closed decision into a pure arm_decision function and drive on_request through it, then unit test the full decision table: arm inside IRR, reject outside IRR, ignore otherwise. This locks the PR's core fail-closed invariant against a fail-open regression at the unit level. The end-to-end arming effect with a real IRR-inserted IterationState stays covered by the functional integration tests. Signed-off-by: Sébastien Han <seb@redhat.com>
praxis-bot review on praxis-proxy#1013 flagged that the IterationState placement guard's accept branch (arm inside an iterative_request_router step) had no unit coverage — only the reject branch did. IterationState cannot be constructed outside praxis-filter (its fields are pub(crate) with no public or test constructor), so the suggested test-only constructor is unavailable without an upstream change. Extract the guard's fail-closed decision into a pure arm_decision function and drive on_request through it, then unit test the full decision table: arm inside IRR, reject outside IRR, ignore otherwise. This locks the PR's core fail-closed invariant against a fail-open regression at the unit level. The end-to-end arming effect with a real IRR-inserted IterationState stays covered by the functional integration tests. Signed-off-by: Sébastien Han <seb@redhat.com>
cf28192 to
fd24f8d
Compare
franciscojavierarceo
left a comment
There was a problem hiding this comment.
the current OpenResponses conformance job is still failing six cases with HTTP 504 upstream timeouts (one local schema case passes), so the required check is not green on this head. please rerun and diagnose the conformance job before merging; if the failures are environmental, make that evidence clear in the check or test setup.
openai_stream_events had a logical_stream flag that exposed an
architectural invariant as an operator choice. A Responses stream that
participates in agentic execution must run inside the iterative request
router; a single inference round is simply a one-round logical stream.
Remove the flag and make the filter always compose the current IRR
execution into one logical Responses stream: normalize response
identity, sequence numbers, and output indices across rounds; suppress
intermediate response.created/terminal lifecycle events and [DONE];
emit exactly one final terminal event and sentinel; keep accumulating
output and usage across model/tool/model rounds; and continue surfacing
loop-terminal failures as SSE errors after the HTTP stream has
committed. response_body_access() is therefore always ReadWrite.
Fail closed when the filter is placed outside an IRR: on_request rejects
a streaming Responses request with a 500 SSE error when no IterationState
is present, as defense in depth. The config keeps deny_unknown_fields, so
any config still setting logical_stream now fails to parse.
Migrate every standalone openai_stream_events example and test into an
IRR pipeline:
- stream-events.yaml and responses-to-chat-completions.yaml wrap the
filter (with proxy/router/load_balancer) in an iterative_request_router
step.
- full-flow.yaml becomes a B-shaped gateway: a pre-IRR bypass carrier
routes WebSocket upgrades and prompts/embeddings/files/vector_stores,
while classified POST /v1/responses create requests flow through an IRR
whose inference step hosts openai_responses_proxy (terminal_streaming:
true) and openai_stream_events. openai_response_store stays pre-IRR so
it persists the accumulated response object on the reverse-order
response path, making streaming responses persistable and retrievable
via GET /v1/responses/{id}.
- compact.yaml, full-flow-agentic.yaml, and mcp-dispatch.yaml drop the
now-inert standalone filter with a comment pointing to the in-IRR
examples.
openai_agentic_loop consumes the always-published logical-stream marker
(renamed stream_events_armed) to keep its fail-closed guard against
unsafe terminal streaming without a finalizer.
Update unit tests, functional integration tests, inference fixtures, and
generated documentation to the always-on/fail-closed contract, and drop
full-flow.yaml from the example-coverage skip allowlist.
Closes praxis-proxy#993
Signed-off-by: Sébastien Han <seb@redhat.com>
praxis-bot review on praxis-proxy#1013 flagged that the IterationState placement guard's accept branch (arm inside an iterative_request_router step) had no unit coverage — only the reject branch did. IterationState cannot be constructed outside praxis-filter (its fields are pub(crate) with no public or test constructor), so the suggested test-only constructor is unavailable without an upstream change. Extract the guard's fail-closed decision into a pure arm_decision function and drive on_request through it, then unit test the full decision table: arm inside IRR, reject outside IRR, ignore otherwise. This locks the PR's core fail-closed invariant against a fail-open regression at the unit level. The end-to-end arming effect with a real IRR-inserted IterationState stays covered by the functional integration tests. Signed-off-by: Sébastien Han <seb@redhat.com>
Main advanced with two changes that predate the IRR-only refactor and were incompatible with it once rebased: - web-search-chat-completions.yaml (praxis-proxy#996/praxis-proxy#920) still set the removed `logical_stream` config field, so the example failed to parse under `deny_unknown_fields`. Drop the field and its stale comment; openai_stream_events always composes the logical stream now. - The example's fail-closed test toggled that deleted field. Model the missing logical-stream finalizer by removing openai_stream_events from the IRR step instead; openai_agentic_loop's `responses.logical_stream` metadata guard still fails closed (500) before dispatch. Rename the helper and test to *_without_stream_events. - openai_mcp_tool_resolve streaming discovery-failure tests (praxis-proxy#991) placed openai_stream_events top-level in resolve_yaml_full_flow_store. Outside an IRR the IRR-only guard now fails closed (500) instead of the expected 200 SSE. The resolver self-delivers the full 200 SSE failure lifecycle and writes response_object directly, so stream_events is unnecessary there; drop it and document why it is intentionally absent. Signed-off-by: Sébastien Han <seb@redhat.com>
…eam-events configs PR praxis-proxy#1007 removed the terminal_streaming field from openai_responses_proxy (deny_unknown_fields now rejects it); the streaming transport auto-selects from the effective "stream" value. These two example configs still carried terminal_streaming: true, so every integration test that loads them failed to parse the pipeline (coverage, test, and codex-websocket jobs). Remove the field and align the adjacent comments with the auto-select behavior. Signed-off-by: Sébastien Han <seb@redhat.com>
Signed-off-by: Sébastien Han <seb@redhat.com>
Signed-off-by: Sébastien Han <seb@redhat.com>
c51ac20 to
1fbc9b8
Compare
The example-deadline commit added `timeout_ms: 360000` to the iterative_request_router in agentic-loop.yaml. The vLLM agentic test helper (_write_agentic_config) already injected `timeout_ms: 300000` after `max_iterations`, so the patched temp config carried two `timeout_ms` keys in the same IRR mapping. serde_yaml rejects that as a duplicate key, so Praxis exited before binding its port and every TestAgenticLoopVLLM / TestStreamingMcpDiscoveryFailureVLLM case errored at startup. Inject only `step_timeout_ms` from the helper and let the config's own `timeout_ms` provide the overall deadline, eliminating the duplicate. Signed-off-by: Sébastien Han <seb@redhat.com>
Summary
openai_stream_eventshad alogical_streamflag that turned an architectural invariant into an operator choice. This removes the flag and makes the filter always compose the current IRR execution into one logical Responses stream — normalizing identity/sequence/output indices, suppressing intermediate lifecycle events and[DONE], emitting exactly one terminal event and sentinel, accumulating output and usage across rounds, and surfacing loop-terminal failures as post-commit SSE errors — while failing closed (500 SSE error) when placed outside aniterative_request_router. Every standalone example and test is migrated into an IRR step, includingfull-flow.yaml, which becomes a B-shaped gateway that keepsopenai_response_storepre-IRR so streaming (stream: true) responses are now persisted and retrievable viaGET /v1/responses/{id}.openai_agentic_loopkeeps its fail-closed guard against unsafe terminal streaming by consuming the now-unconditional logical-stream marker, and unit tests, functional integration tests, inference fixtures, and generated docs are updated to the always-on/fail-closed contract.Related issue
Closes #993
Validation
cargo test -p praxis-ai-apis(2926 passed + 4 doctests, incl. the rewrittenstream_eventssuite)cargo test -p praxis-tests-integration --test suite examples(276),session_replay(9),codex_websocket(2),make test-inference-fixtures(all replay)make lint(alsomake buildandmake docwith rustdoc-D warnings)Checklist
Signed-off-bytrailer.Breaking changes
Any config still setting
openai_stream_events.logical_streamnow fails to parse (deny_unknown_fields) — remove the flag, as the filter always composes.openai_stream_eventsmust run inside aniterative_request_routerstep: astream: trueResponses request with the filter placed outside an IRR is rejected with a 500 SSE error. Migrate standalone placements into an IRR step (seestream-events.yamlandagentic-loop.yaml).