fix(parser): fail closed on multiple concatenated tool envelopes - #1788
Open
brkastner wants to merge 1 commit into
Open
fix(parser): fail closed on multiple concatenated tool envelopes#1788brkastner wants to merge 1 commit into
brkastner wants to merge 1 commit into
Conversation
Codex OAuth in Responses mode emitted two concatenated top-level JSON
roots. extract_tool_request() returned None (root != content) and
is_misformatted_tool_request() returned False, so the Responses branch in
agent.py wrapped the raw tool protocol as response(text=...) and rendered
it to the user verbatim.
Classify multiple top-level roots as misformatted when at least one root
parses as a tool request. Nothing is selected or executed, so the
existing repair warning and unusable-response circuit breaker handle the
turn. No behaviour change for single envelopes, literal }{ inside
strings, prose, fenced or truncated JSON.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
When a model emits two concatenated top-level JSON tool envelopes in one turn, Agent Zero renders the raw tool protocol to the user as if it were a chat message. Nothing executes, no repair warning fires, and the unusable-response circuit breaker never counts the turn.
Observed with the Codex OAuth provider in Responses mode (
gpt-5.6-sol), where the assistant produced:The two objects are adjacent with no separator. The user sees both JSON blobs verbatim in the chat window.
Root cause
extract_json_root_strings()correctly enumerates every top-level JSON root. The consumers do not.extract_tool_request()requiresroot == content.strip(). With a second envelope present that equality fails, so it returnsNone.is_misformatted_tool_request()has no notion of multiple roots, so it also returnsFalse.That combination is a silent fail-open. In
agent.pythe Responses path treats "no tool request and not misformatted" as ordinary prose and wraps it asresponse(text=...), so raw tool protocol reaches the user.process_tools()never reaches thefw.msg_misformat.mdbranch, soStopUnusableResponseLoopnever increments_unusable_response_failures. A model that repeats the behaviour is not stopped.Fix
Classify multiple top-level roots as misformatted when at least one root parses as a tool request.
No root is selected and none is executed. Picking the first envelope would run an action the model did not request in isolation, so this fails closed instead. Routing through the existing misformat path means the current repair warning and circuit breaker handle the turn with no new prompt and no new state.
The change is additive: one new function plus a five-line guard in
is_misformatted_tool_request().extract_tool_request(),extract_json_root_string(),json_parse_dirty(),extract_json_root_strings(), and_is_tool_request()are untouched.Behaviour change
Measured against the pre-existing public API.
RENDERS_RAWis the fail-open condition inagent.pythat leaks tool protocol into the chat.}{inside a string valueOnly the two broken cases move.
Tests
Adds
tests/test_multiple_tool_roots.py, 15 tests: the concatenation cases, the circuit-breaker path throughStopUnusableResponseLoopat the configured limit, Responses mode not wrapping concatenated protocol as text, and over-trigger guards for literal braces inside strings, nested braces, prose, truncated input, non-string input, and multiple non-tool roots.Full suite, same container image, stock vs patched:
Identical failure set, plus the 15 new tests. The 60 failures and 12 collection errors reproduce on unmodified
mainand are unrelated to this change.Note on the provider side
output_text()inhelpers/litellm_transport.pyjoins text across all message items and content blocks with"".join(pieces). If a provider returns two items that each contain a complete envelope, that join produces exactly this shape with no delimiter. I have not changed it, since the correct separator depends on provider semantics and the parser should fail closed regardless. Flagging it as a likely upstream contributor.