Skip to content

Recorder saves successful non-streamed Responses API replies as error fixtures #615

Closed

Description

@contextablemark

Summary

The recorder treats a successful non-streamed OpenAI Responses API reply (POST /v1/responses, stream: false, HTTP 200) as an unknown format. In record mode it saves the reply as an error fixture. Replaying that fixture returns proxy_error for a request that succeeded upstream.

Where

src/recorder.ts, in buildFixtureResponse:

  • The Array.isArray(obj.output) branch only recognizes a Responses body when it carries native citation metadata. The comment says: "an uncited body … keeps the error fallback below, as before citations."
  • Every other Responses body falls through to the final fallback, which logs Could not detect response format from upstream (status=200) — saving as error fixture and returns { error: { message: "Could not detect response format from upstream", type: "proxy_error" }, status }.

Observed

Seen on 2026-11-11 in the showcase prod aimock, which runs --proxy-only --fixtures /fixtures. The showcase openai-agents-python subagents demo calls Runner.run (non-streamed), and that produced three warnings like this one:

[aimock] Could not detect response format from upstream (status=200) — saving as error fixture; top-level keys: [id, object, created_at, status, access_programs, background, billing, completed_at, err…

The user-visible result was correct, because proxy-only mode passes the upstream body through and returns { kind: "skipped" } before persisting anything (around recorder.ts:358). So:

  • proxy-only: no fixture is written. The warning is misleading noise and says "saving" when nothing is saved.
  • record mode (--record): a real error fixture is written for a successful turn. A later replay, such as a recorded D5/D6 suite, fails with proxy_error instead of replaying the model's reply.

Expected

  • A non-streamed Responses body with object: "response" and an output array is collapsed into a normal fixture, with text, tool calls and reasoning where present, whether or not it has citations. The streamed recorder already handles the same turn.
  • An unknown 2xx body should not become an error fixture. Possible alternatives: skip it, or save it as raw json with a warning.
  • In proxy-only mode, the warning should not say "saving as error fixture".

Repro sketch

  1. aimock --record --provider-openai https://api.openai.com
  2. POST /v1/responses with {"model": "<any>", "input": "Say hi", "stream": false}, which returns 200 upstream.
  3. The recorded fixture contains error.type = "proxy_error". Replaying the same request returns that error instead of the reply.

Context

Found during the showcase openai-agents-python live-provider check. Agents SDK sub-agent tools and other non-streamed Runner.run calls hit this path routinely.

🤖 Generated with Claude Code

Activity

  1. added a commit that references this issue on Nov 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions