feat(anthropic): stream the Messages web-search terminal answer - #1012
Merged
Conversation
…agentic Rename the buffered Anthropic Messages web-search example config and its functional integration test suite to the full-flow-agentic naming, mirroring the OpenAI Responses full-flow-agentic example and establishing a parallel convention across providers. - examples/configs/anthropic/messages-web-search.yaml -> full-flow-agentic.yaml (listener anthropic-full-flow-agentic, filter chain full-flow-agentic) - tests/integration suite anthropic_messages_web_search.rs -> anthropic_full_flow_agentic.rs (config path + module registration) - repoint the filter's live-demo doc comment to the new config path - regenerate docs/filters/anthropic_web_search.md and examples/README.md Signed-off-by: Sébastien Han <seb@redhat.com>
Add stream: true support to the server-owned Anthropic Messages web-search loop. The anthropic_web_search filter gains a terminal_streaming knob that selects Praxis's streaming subrequest transport per request from the client's stream flag, delivering the terminal inference response incrementally as one coherent client-visible SSE lifecycle across IRR re-entry while intermediate model/search transitions stay internal and the managed WebSearch tool-use block is suppressed. Buffered (stream: false) requests keep the existing JSON path, so one pipeline serves both modes. - apis/src/anthropic/web_search/streaming.rs: LogicalStream cross-round SSE transform (single message_start, suppressed WebSearch blocks, deferred per-round message_delta/message_stop, terminal usage aggregation; fails closed to one terminal error event on any malformed or incomplete round) - apis/src/anthropic/web_search/mod.rs: streaming route, StreamTermination handling, Accept-Encoding strip, terminal_streaming trait overrides - apis/src/web_search/config.rs: terminal_streaming field (anthropic-only; openai_web_search forces it false) - examples/configs/anthropic/full-flow-agentic.yaml: terminal_streaming: true serves both buffered and streaming clients from one pipeline - unified integration suite (20 tests = 7 buffered + 13 streaming) and regenerated filter/example docs Refs praxis-proxy#649 Signed-off-by: Sébastien Han <seb@redhat.com>
leseb
force-pushed
the
leseb/issue-649
branch
from
September 8, 2026 16:08
460dbba to
d63dd2d
Compare
praxis-bot
reviewed
Sep 8, 2026
praxis-bot
left a comment
praxis-bot
left a comment
There was a problem hiding this comment.
praxis-bot review: feat(anthropic): stream the Messages web-search terminal answer
This is a well-crafted addition. The fail-closed security posture is consistent throughout -- every malformed frame, suppression mismatch, truncated lifecycle, and transport fault surfaces as exactly one terminal error event with no raw upstream bytes leaking into the transformed stream. The UntransformableRound marker design correctly bridges the response-header phase (where status and encoding are visible) and the streaming body phase (where they are not), and the stale-marker cleanup prevents cross-round leaks. Test coverage is thorough: 230+ unit tests for the streaming module, 20 integration tests exercising both modes from one example config, and explicit coverage of the open-block / stray-stop / duplicate-stop completeness gate.
One medium finding below.
decode_utf8 allocated a fresh String for every streamed SSE chunk, even on the hot path where the chunk is already valid UTF-8 and no tail is retained. Return Cow<'chunk, str> instead: a well-formed chunk is validated once and borrows through to line_buffer with no intermediate heap allocation, and only the rare split-code-point case (joining a retained tail with the chunk) allocates a single owned buffer, which is moved out by value rather than copied again. Extract the incomplete-tail retention into retain_incomplete_tail so both the borrowed and owned paths share one fail-closed boundary (genuinely invalid UTF-8, an incomplete sequence at end-of-stream, and an over-long retained tail). Behavior is unchanged; a new characterization test locks the retained-tail re-stash across a four-byte code point delivered one byte per chunk. Addresses the code-review finding on praxis-proxy#1012. Signed-off-by: Sébastien Han <seb@redhat.com>
leseb
enabled auto-merge
September 9, 2026 10:03
franciscojavierarceo
approved these changes
Sep 9, 2026
franciscojavierarceo
left a comment
franciscojavierarceo
left a comment
Contributor
There was a problem hiding this comment.
reviewed the current head; no actionable findings.
leseb
added a commit
to leseb/praxis-ai
that referenced
this pull request
Sep 9, 2026
Resolve test-only conflicts from praxis-proxy#1004 (streaming previous_response_id restore), praxis-proxy#1026 (audio/resource-link MCP results), and praxis-proxy#1012: - rehydrate/tests.rs: union the imports — keep origin's openai::sse {SseFrame, SseFrameParser} and this branch's PendingApprovalRecord. - test_openai_responses_vllm.py MCPHandler tools/call: keep the approval round-trip tool-call counter and adopt praxis-proxy#1026's get_weather_map resource_link (non-text) content branch. - mcp_dispatch/tests.rs: pass None for the approval_request_id argument praxis-proxy#1026's process_call_result test did not supply. Signed-off-by: Sébastien Han <seb@redhat.com>
leseb
added a commit
to leseb/praxis-ai
that referenced
this pull request
Sep 9, 2026
…#1026) into leseb/fix-batched-tool-calls Reconcile the latest main with the batched/parallel model tool-call support on this branch. - vLLM SDK test mock (test_openai_responses_vllm.py): serve get_weather, get_time (this branch's two-tool batch test) and get_weather_map (praxis-proxy#1026's non-text resource_link test) together, dispatching each in the tools/call handler so both scenarios run against one mock. - mcp_dispatch tests: pass this branch's max_result_bytes cap argument to content_blocks_to_output and process_call_result in praxis-proxy#1026's new audio/resource-link/complex-schema coverage. Signed-off-by: Sébastien Han <seb@redhat.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
stream: truesupport to the server-owned Anthropic Messages web-searchloop, so a single gateway serves both streaming and buffered clients from one
pipeline. Mirrors the OpenAI Responses IRR terminal-streaming design (#756).
Per review discussion this lands as one PR with two commits — the streaming
feature plus a small example rename that was originally split out:
refactor(examples)— rename the bufferedmessages-web-search.yamlexampleand its integration suite to
full-flow-agentic.yaml/anthropic_full_flow_agentic.rs, establishing an Anthropic full-flow-agenticnaming parallel to
openai/responses/full-flow-agentic.yaml.feat(anthropic)— the streaming implementation.What changed
anthropic_web_searchgains aterminal_streamingknob (anthropic-only;openai_web_searchforces it false). When enabled and the client sentstream: true, the terminal inference response is streamed incrementally asone coherent Anthropic Messages SSE lifecycle (single
message_start,forwarded text
content_block_*frames, then one terminalmessage_delta/message_stop) across IRR re-entry. Intermediate model/search transitions stayinternal and the managed
WebSearchtool-use block is suppressed.stream: falserequests keep the existing buffered JSON path — one pipeline,both modes, selected per request from the client
streamflag.apis/src/anthropic/web_search/streaming.rs:LogicalStreamcross-roundSSE transform (running output-block-index offset, deferred per-round
message_delta/message_stop, terminal usage aggregation). Fails closed toa single terminal
errorevent on any malformed, content-encoded,incomplete-lifecycle, or mid-stream-terminated round.
apis/src/anthropic/web_search/mod.rs: streaming route,StreamTerminationhandling,
Accept-Encodingstrip,terminal_streamingtrait overrides.examples/configs/anthropic/full-flow-agentic.yamlsetsterminal_streaming: trueand documents both modes.Testing
anthropic_full_flow_agentic.rs: 20 integrationtests (7 buffered + 13 streaming) — round-trip re-entry, provider failure,
caller-header preservation, sequential searches, state/body caps, incremental
delivery before upstream completion, fragmented reassembly, fail-closed on
malformed frame / premature clean EOF / mid-stream termination / later-round
non-success, iteration ceiling, and downstream cancellation.
web_searchunit tests.make lintandmake docclean; regenerated filter/example docs.Closes #649