Skip to content

fix(responses): keep attachment-only user items text-less for userMessage matching - #611

Merged
contextablemark merged 3 commits into
mainfrom
fix/responses-attachment-only-user-text
Nov 10, 2026
Merged

contextablemark merged 3 commits into
mainfrom
fix/responses-attachment-only-user-text

Conversation

@contextablemark

@contextablemark contextablemark commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Some SDKs send a single multimodal user turn to /v1/responses as two consecutive user items: the prompt as input_text, then the attachment alone as input_image or input_file. AG2 1.1.2's OpenAI Responses mapper does this (ag2/config/openai/mappers.py).

getLastUserText already skips a trailing user message with no text, for this exact split on the Chat Completions path. But the Responses converter flattened the attachment-only item to "". The guard deliberately treats "" as real text, so userMessage fixtures keyed on the prompt never matched, and strict mode returned 503.

Attachment-only user items now convert to content: null, the same text-less shape the Chat Completions path produces. An explicit empty input_text still converts to "". System, assistant and tool-output conversion are unchanged.

Impact

This makes the CopilotKit showcase's ag2 multimodal cells red on staging (CopilotKit PNI-796). Both the image and PDF prompts get a 503 "no fixture matched".

aimock split turn (AG2 shape) same content as one item
1.37.4 503 200
1.47.0 503 200
this branch 200 200

Test plan

  • New unit tests in responsesInputToMessages: attachment-only items become null, and an explicit empty input_text stays "".
  • New strict-mode end-to-end tests: a prompt followed by an input_image item, and by an input_file item, both match their fixtures. These failed with 503 before the fix.
  • pnpm run format:check, pnpm run lint, pnpm run typecheck, pnpm build, and pnpm api:report (no report change).
  • pnpm run test: 20,064 passed. The one failure is response-a9-equivalence, which fails the same way on main on this machine with "A9 hard-ceiling witness requires 2 GiB free memory".
  • After release: bump the showcase aimock pin and run a local D6 probe for ag2:multimodal to confirm red→green.

🤖 Generated with Claude Code

…sage matching

AG2's OpenAI Responses mapper sends one multimodal turn as a text-only user
item followed by an attachment-only one. The Responses converter flattened the
attachment-only item to "", which getLastUserText treats as real text, so
userMessage fixtures keyed on the prompt never matched and strict mode
returned 503. Such items now convert to null content, matching how the Chat
Completions path already handles the same split.
@pkg-pr-new

pkg-pr-new Bot commented Nov 10, 2026 •

Copy link
Copy Markdown

Open in StackBlitz

npm i https://pkg.pr.new/@copilotkit/aimock@611

commit: 1476c7a

@contextablemark
contextablemark marked this pull request as ready for review November 10, 2026 23:57

@jpr5 jpr5 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good: attachment-only Responses user items now convert to the same text-less shape as the Chat Completions path, with strict-mode end-to-end tests for the image and file cases.

@contextablemark
contextablemark merged commit f9c9955 into main Nov 10, 2026
29 checks passed
@contextablemark
contextablemark deleted the fix/responses-attachment-only-user-text branch November 10, 2026 23:58
@jpr5 jpr5 mentioned this pull request Nov 11, 2026
jpr5 added a commit that referenced this pull request Nov 11, 2026
**Merging this publishes @copilotkit/aimock 1.48.0 to npm and
aimock-pytest 0.7.2 to PyPI.**

## [1.48.0] - 2026-11-10

### Added

- A2A `onConversation()` scripts multi-turn tasks that ask for input,
then complete (#591)
- Record and replay native citations and annotations from real provider
turns (#594)
- Record and replay BytePlus Seed Speech TTS, ElevenLabs Scribe and
Audio Isolation (#601)

### Changed

- `aimock-pytest` 0.7.2 defaults to `@copilotkit/aimock` 1.48.0

### Fixed

- A2A Get/CancelTask return task as `result`; split Responses attachment
turns match (#595, #611)

## Version carriers (from `git grep -F 1.47.0`)

Bumped to 1.48.0:
- `package.json` version
- `charts/aimock/Chart.yaml` appVersion
- `.claude-plugin/plugin.json` version
- `.claude-plugin/marketplace.json` source version (`^1.48.0`)
- `docs/index.html` banner
- `packages/aimock-pytest/README.md` `--aimock-version` default
- `packages/aimock-pytest/src/aimock_pytest/_version.py`
`AIMOCK_VERSION`

Also bumped: `packages/aimock-pytest/pyproject.toml` 0.7.1 → 0.7.2,
because the pin moved (same pattern as #589).

Left as is (not carriers): the `## [1.47.0]` CHANGELOG heading, the
three "New in 1.47.0" notes in `docs/docs/index.html`, and the three
"1.47.0+" notes in the `docs/index.html` comparison matrix. These mark
when a feature first shipped.

## README

#601 added docs to the speech page but not to the README. This PR adds:
- Seed Speech, Scribe and Audio Isolation to the LLMock row and the
Multimedia APIs line
- `AIMOCK_PROVIDER_BYTEPLUS_SPEECH_KEY` to the upstream-key table

The CHANGELOG `#600` reference is corrected to `#601`. #600 is not a PR.

The other features already have docs: A2A conversations (#591), citation
recording (#594) and the GetTask result shape (#595). The package.json
description is synced from the README subtitle on publish.

## Local checks
- `pnpm install --frozen-lockfile`, `format:check`, `lint`, `typecheck`,
`build`: pass
- `api:report`: 14 entries, 0 failed, no report change
- `vitest run`: 488 files, 20,073 passed, 11 skipped (includes the docs
tests)
- `pytest` in `packages/aimock-pytest` against the local build: 251
passed
@linear-code

linear-code Bot commented Nov 11, 2026

Copy link
Copy Markdown

PNI-796

JadeCong pushed a commit to CloudEngineHub/CopilotKit that referenced this pull request Nov 11, 2026
AG2's OpenAI Responses mapper sends a prompt and its attachment as two user
items. aimock before 1.48.0 read the attachment-only item as empty text, so the
ag2 multimodal fixtures never matched and strict mode returned 503
(CopilotKit/aimock#611, PNI-796).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
JadeCong pushed a commit to CloudEngineHub/CopilotKit that referenced this pull request Nov 11, 2026
…pilotKit#8324)

## Summary

This bumps the showcase aimock base image to **1.48.0**
(`sha256:51d8e426…`, from `sha256:fcc50103…`) so the `ag2` multimodal
cells can go green on staging. It is a one-line change to
`showcase/aimock/Dockerfile`.

**Cause.** AG2 1.1.2's OpenAI Responses mapper sends one turn with an
attachment as two user items: the prompt as `input_text`, then the
attachment alone as `input_image` or `input_file`. aimock before 1.48.0
read the attachment-only item as empty text instead of no text, so it
never fell back to the prompt. The `ag2` multimodal fixtures never
matched, and strict mode returned 503 "no fixture matched". The fix is
CopilotKit/aimock#611, released in v1.48.0. This closes PNI-796.

The 1.48.0 release also includes the other unreleased aimock changes
since 1.47.0: A2A `onConversation()`, citation record/replay, the audio
record/replay routes, and an A2A result-shape fix. See the aimock
CHANGELOG.

## Evidence

The same isolated local stack, probe and fixtures were used for both
runs. Only the aimock image changed.

| aimock image | `bin/showcase test ag2:multimodal --d6 --direct` |
aimock strict misses |
|---|---|---|
| current pin `fcc50103…` | red: `InternalServerError` banner, 0 turns |
6 |
| 1.48.0 `51d8e426…` | **green**: 2 turns (image + PDF), 7.8s | 0 |

The production image builds from this Dockerfile: the Node donor check
passes (v22.23.4), and `/app/package.json` reports 1.48.0.

Red→green cells: `d6:ag2/multimodal`, proven locally.
`d5:ag2/multimodal` failed on staging with the same 503, so it should
also clear, but I have not verified it.

## Not in this PR

`showcase/scripts/package.json` still pins `@copilotkit/aimock` 1.37.4
for the local/CI frontend-matrix runner. Bumping it changes the lockfile
and moves every matrix cell up 11 minor versions, so it should be a
separate PR.

## After merge

- Check the `showcase-aimock` staging deploy picked up the new image,
then read PocketBase `d6:ag2/multimodal` (and D5) for a fresh green
probe recorded after the deploy.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
contextablemark added a commit to CopilotKit/CopilotKit that referenced this pull request Nov 11, 2026
The frontend-matrix, on-demand e2e, integration-docs and Mastra remote A2UI
workflows run llmock from showcase/scripts. 1.37.4 lacks CopilotKit/aimock#611,
so the AG2 multimodal cell fails there with a strict no-fixture-match 503,
the same bug #8324 fixed for the staging aimock image.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants