Skip to content

refactor(responses): make openai_stream_events IRR-only - #1013

Open
leseb wants to merge 7 commits into
praxis-proxy:mainfrom
leseb:leseb/issue-993-implement
Open

refactor(responses): make openai_stream_events IRR-only#1013
leseb wants to merge 7 commits into
praxis-proxy:mainfrom
leseb:leseb/issue-993-implement

Conversation

@leseb

@leseb leseb commented Sep 8, 2026

Copy link
Copy Markdown
Collaborator

Summary

openai_stream_events had a logical_stream flag that turned an architectural invariant into an operator choice. This removes the flag and makes the filter always compose the current IRR execution into one logical Responses stream — normalizing identity/sequence/output indices, suppressing intermediate lifecycle events and [DONE], emitting exactly one terminal event and sentinel, accumulating output and usage across rounds, and surfacing loop-terminal failures as post-commit SSE errors — while failing closed (500 SSE error) when placed outside an iterative_request_router. Every standalone example and test is migrated into an IRR step, including full-flow.yaml, which becomes a B-shaped gateway that keeps openai_response_store pre-IRR so streaming (stream: true) responses are now persisted and retrievable via GET /v1/responses/{id}. openai_agentic_loop keeps its fail-closed guard against unsafe terminal streaming by consuming the now-unconditional logical-stream marker, and unit tests, functional integration tests, inference fixtures, and generated docs are updated to the always-on/fail-closed contract.

Related issue

Closes #993

Validation

  • Unit tests — cargo test -p praxis-ai-apis (2926 passed + 4 doctests, incl. the rewritten stream_events suite)
  • Integration or functional tests — cargo test -p praxis-tests-integration --test suite examples (276), session_replay (9), codex_websocket (2), make test-inference-fixtures (all replay)
  • make lint (also make build and make doc with rustdoc -D warnings)

Checklist

  • I reviewed every changed line and can explain the change.
  • New capabilities include an example config and functional example test.
  • User-facing behavior and generated documentation are updated.
  • Performance-sensitive changes include appropriate benchmark or load-test evidence. — N/A; this refactor removes a flag and adds no perf-sensitive path.
  • Commits are signed and include a Signed-off-by trailer.

Breaking changes

Any config still setting openai_stream_events.logical_stream now fails to parse (deny_unknown_fields) — remove the flag, as the filter always composes. openai_stream_events must run inside an iterative_request_router step: a stream: true Responses request with the filter placed outside an IRR is rejected with a 500 SSE error. Migrate standalone placements into an IRR step (see stream-events.yaml and agentic-loop.yaml).

@leseb
leseb requested review from a team and franciscojavierarceo September 8, 2026 16:16

@praxis-bot praxis-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

praxis-bot review: refactor(responses): make openai_stream_events IRR-only

Clean refactor that hardens the contract: openai_stream_events now always composes logical streams and must live inside an iterative_request_router step. The logical_stream config flag is removed, the fail-closed guard rejects misconfigured placements, and all examples/tests/docs are updated accordingly. The architectural invariant is well-enforced and the test coverage (both unit and functional integration) is thorough.

One medium finding below.


Reviewed files: 23/23
Classification: Medium (1)

Comment thread apis/src/openai/responses/stream_events/tests.rs
leseb added a commit to leseb/praxis-ai that referenced this pull request Sep 9, 2026
praxis-bot review on praxis-proxy#1013 flagged that the IterationState placement
guard's accept branch (arm inside an iterative_request_router step) had
no unit coverage — only the reject branch did. IterationState cannot be
constructed outside praxis-filter (its fields are pub(crate) with no
public or test constructor), so the suggested test-only constructor is
unavailable without an upstream change.

Extract the guard's fail-closed decision into a pure arm_decision
function and drive on_request through it, then unit test the full
decision table: arm inside IRR, reject outside IRR, ignore otherwise.
This locks the PR's core fail-closed invariant against a fail-open
regression at the unit level. The end-to-end arming effect with a real
IRR-inserted IterationState stays covered by the functional integration
tests.

Signed-off-by: Sébastien Han <seb@redhat.com>
leseb added a commit to leseb/praxis-ai that referenced this pull request Sep 9, 2026
praxis-bot review on praxis-proxy#1013 flagged that the IterationState placement
guard's accept branch (arm inside an iterative_request_router step) had
no unit coverage — only the reject branch did. IterationState cannot be
constructed outside praxis-filter (its fields are pub(crate) with no
public or test constructor), so the suggested test-only constructor is
unavailable without an upstream change.

Extract the guard's fail-closed decision into a pure arm_decision
function and drive on_request through it, then unit test the full
decision table: arm inside IRR, reject outside IRR, ignore otherwise.
This locks the PR's core fail-closed invariant against a fail-open
regression at the unit level. The end-to-end arming effect with a real
IRR-inserted IterationState stays covered by the functional integration
tests.

Signed-off-by: Sébastien Han <seb@redhat.com>
@leseb
leseb force-pushed the leseb/issue-993-implement branch from cf28192 to fd24f8d Compare September 9, 2026 11:00

@franciscojavierarceo franciscojavierarceo left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the current OpenResponses conformance job is still failing six cases with HTTP 504 upstream timeouts (one local schema case passes), so the required check is not green on this head. please rerun and diagnose the conformance job before merging; if the failures are environmental, make that evidence clear in the check or test setup.

openai_stream_events had a logical_stream flag that exposed an
architectural invariant as an operator choice. A Responses stream that
participates in agentic execution must run inside the iterative request
router; a single inference round is simply a one-round logical stream.

Remove the flag and make the filter always compose the current IRR
execution into one logical Responses stream: normalize response
identity, sequence numbers, and output indices across rounds; suppress
intermediate response.created/terminal lifecycle events and [DONE];
emit exactly one final terminal event and sentinel; keep accumulating
output and usage across model/tool/model rounds; and continue surfacing
loop-terminal failures as SSE errors after the HTTP stream has
committed. response_body_access() is therefore always ReadWrite.

Fail closed when the filter is placed outside an IRR: on_request rejects
a streaming Responses request with a 500 SSE error when no IterationState
is present, as defense in depth. The config keeps deny_unknown_fields, so
any config still setting logical_stream now fails to parse.

Migrate every standalone openai_stream_events example and test into an
IRR pipeline:

- stream-events.yaml and responses-to-chat-completions.yaml wrap the
  filter (with proxy/router/load_balancer) in an iterative_request_router
  step.
- full-flow.yaml becomes a B-shaped gateway: a pre-IRR bypass carrier
  routes WebSocket upgrades and prompts/embeddings/files/vector_stores,
  while classified POST /v1/responses create requests flow through an IRR
  whose inference step hosts openai_responses_proxy (terminal_streaming:
  true) and openai_stream_events. openai_response_store stays pre-IRR so
  it persists the accumulated response object on the reverse-order
  response path, making streaming responses persistable and retrievable
  via GET /v1/responses/{id}.
- compact.yaml, full-flow-agentic.yaml, and mcp-dispatch.yaml drop the
  now-inert standalone filter with a comment pointing to the in-IRR
  examples.

openai_agentic_loop consumes the always-published logical-stream marker
(renamed stream_events_armed) to keep its fail-closed guard against
unsafe terminal streaming without a finalizer.

Update unit tests, functional integration tests, inference fixtures, and
generated documentation to the always-on/fail-closed contract, and drop
full-flow.yaml from the example-coverage skip allowlist.

Closes praxis-proxy#993

Signed-off-by: Sébastien Han <seb@redhat.com>
praxis-bot review on praxis-proxy#1013 flagged that the IterationState placement
guard's accept branch (arm inside an iterative_request_router step) had
no unit coverage — only the reject branch did. IterationState cannot be
constructed outside praxis-filter (its fields are pub(crate) with no
public or test constructor), so the suggested test-only constructor is
unavailable without an upstream change.

Extract the guard's fail-closed decision into a pure arm_decision
function and drive on_request through it, then unit test the full
decision table: arm inside IRR, reject outside IRR, ignore otherwise.
This locks the PR's core fail-closed invariant against a fail-open
regression at the unit level. The end-to-end arming effect with a real
IRR-inserted IterationState stays covered by the functional integration
tests.

Signed-off-by: Sébastien Han <seb@redhat.com>
Main advanced with two changes that predate the IRR-only refactor and
were incompatible with it once rebased:

- web-search-chat-completions.yaml (praxis-proxy#996/praxis-proxy#920) still set the removed
  `logical_stream` config field, so the example failed to parse under
  `deny_unknown_fields`. Drop the field and its stale comment;
  openai_stream_events always composes the logical stream now.

- The example's fail-closed test toggled that deleted field. Model the
  missing logical-stream finalizer by removing openai_stream_events from
  the IRR step instead; openai_agentic_loop's `responses.logical_stream`
  metadata guard still fails closed (500) before dispatch. Rename the
  helper and test to *_without_stream_events.

- openai_mcp_tool_resolve streaming discovery-failure tests (praxis-proxy#991)
  placed openai_stream_events top-level in resolve_yaml_full_flow_store.
  Outside an IRR the IRR-only guard now fails closed (500) instead of the
  expected 200 SSE. The resolver self-delivers the full 200 SSE failure
  lifecycle and writes response_object directly, so stream_events is
  unnecessary there; drop it and document why it is intentionally absent.

Signed-off-by: Sébastien Han <seb@redhat.com>
…eam-events configs

PR praxis-proxy#1007 removed the terminal_streaming field from openai_responses_proxy
(deny_unknown_fields now rejects it); the streaming transport auto-selects
from the effective "stream" value. These two example configs still carried
terminal_streaming: true, so every integration test that loads them failed
to parse the pipeline (coverage, test, and codex-websocket jobs). Remove the
field and align the adjacent comments with the auto-select behavior.

Signed-off-by: Sébastien Han <seb@redhat.com>
Signed-off-by: Sébastien Han <seb@redhat.com>
Signed-off-by: Sébastien Han <seb@redhat.com>
@leseb
leseb force-pushed the leseb/issue-993-implement branch from c51ac20 to 1fbc9b8 Compare September 9, 2026 13:45
The example-deadline commit added `timeout_ms: 360000` to the iterative_request_router in agentic-loop.yaml. The vLLM agentic test helper (_write_agentic_config) already injected `timeout_ms: 300000` after `max_iterations`, so the patched temp config carried two `timeout_ms` keys in the same IRR mapping. serde_yaml rejects that as a duplicate key, so Praxis exited before binding its port and every TestAgenticLoopVLLM / TestStreamingMcpDiscoveryFailureVLLM case errored at startup.

Inject only `step_timeout_ms` from the helper and let the config's own `timeout_ms` provide the overall deadline, eliminating the duplicate.

Signed-off-by: Sébastien Han <seb@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

refactor(responses): make openai_stream_events IRR-only

3 participants