Skip to content

fix(translator): preserve Claude URL images for Codex (GPT-5.4) vision#2931

Open
dos1989 wants to merge 1 commit into
router-for-me:devfrom
dos1989:fix/8317-claude-url-image-gpt54
Open

fix(translator): preserve Claude URL images for Codex (GPT-5.4) vision#2931
dos1989 wants to merge 1 commit into
router-for-me:devfrom
dos1989:fix/8317-claude-url-image-gpt54

Conversation

@dos1989
Copy link
Copy Markdown

@dos1989 dos1989 commented Apr 21, 2026

Summary

  • Claude→Codex translator (internal/translator/codex/claude/codex_claude_request.go) silently dropped Claude image content whose source.type == "url" because it only read source.data / source.base64. Claude clients that send URL-referenced images to gpt-5.4 ended up with text-only context — model couldn't 睇圖.
  • Extracted a claudeImageSourceToDataURL helper that handles both url and base64 sources (matching the existing internal/translator/openai/claude/openai_claude_request.go behaviour), and routed the top-level user-image path and the nested tool_result image path through it.
  • Added TestConvertClaudeRequestToCodex_ImageSources covering base64 + URL cases.

Root cause

codex_claude_request.go handled image content at two call sites (top-level user message + tool_result.content[]) but only read source.data / source.base64. Anthropic's API also allows {"type":"image","source":{"type":"url","url":"..."}}, which was silently skipped — no error surfaced upstream, the request simply reached Codex with text only.

Verification

  • go build ./...
  • go vet ./internal/translator/codex/claude/...
  • go test ./internal/translator/codex/claude/...
  • Live E2E against a local build on 127.0.0.1:8318: Claude /v1/messages with source.type="url" image + model: gpt-5.4 → response: "The image shows the GitHub logo, a black Octocat silhouette in a circle." (before the fix, the model saw text only and described no image).

Test plan

  • Base64 source still produces input_image with a data:<mime>;base64,... URL
  • URL source produces input_image with the original URL
  • go test ./internal/translator/codex/claude/... passes
  • Live E2E via Claude-format client on gpt-5.4

Out of scope (follow-ups)

Discovered during review, not addressed here to keep the fix minimal:

  1. source.type == "file" (Anthropic file_id) still returns empty from the helper — both call sites will skip it. Requires the helper to return a structured image part (image_url vs file_id), not a string.
  2. tool_result.content as a single image object (non-array) falls through to the string path. Sibling openai_claude_request.go already handles this shape.
  3. internal/translator/antigravity/claude/antigravity_claude_request.go has the same URL-source gap (base64 only at lines 281/317/347) and its existing tests lock that behaviour.

Happy to split these into separate PRs if the project prefers.

@github-actions github-actions Bot changed the base branch from main to dev April 21, 2026 01:45
@github-actions
Copy link
Copy Markdown

This pull request targeted main.

The base branch has been automatically changed to dev.

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the handling of Claude image sources by introducing a helper function claudeImageSourceToDataURL. This helper consolidates the logic for processing both base64-encoded and URL-referenced image sources, reducing code duplication in ConvertClaudeRequestToCodex. Additionally, new test cases were added to verify the correct conversion of both image source types. I have no feedback to provide.

Copy link
Copy Markdown
Collaborator

@luispater luispater left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Fixes Claude→Codex request translation dropping source.type == "url" image parts by centralizing image-source conversion and reusing it for both user message images and tool_result image content. Adds unit tests for base64 and URL sources.

Findings

Blocking

  • None.

Non-blocking

  • Consider renaming claudeImageSourceToDataURL to ...ImageURL since it can return a plain URL.
  • Consider adding coverage for the tool_result.content[].type == "image" path (now also uses the helper).
  • FYI this PR will keep failing translator-path-guard due to touching internal/translator/... unless the merge policy/check is overridden.

Test plan

  • Not run locally (no branch checkout per request).
  • CI pr-test-build is green; recommend also running go test ./internal/translator/codex/claude/....

This is an automated Codex review result and still requires manual verification by a human reviewer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants