feat(codex): add Azure OpenAI image generation support with direct routing#2960
feat(codex): add Azure OpenAI image generation support with direct routing#2960zilianpn wants to merge 1 commit into
Conversation
zilianpn
force-pushed
the
feature/azure-image-direct
branch
from
625d630 to
70f969e
Compare
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a direct execution path for image generation and editing models to bypass the standard Responses API orchestration. This change is specifically designed to support providers like Azure OpenAI, where models are deployed to unique URLs that do not support cross-deployment tool routing. The feedback provided focuses on generalizing the implementation to support both generation and editing endpoints dynamically, improving context propagation by using the request context, and ensuring compatibility with non-Azure providers by making the removal of the response_format parameter conditional. Additionally, it is recommended to use consistent error-handling utilities for better status reporting.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 625d63026c
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
zilianpn
force-pushed
the
feature/azure-image-direct
branch
from
70f969e to
5e4c2d3
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5e4c2d3f93
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
zilianpn
force-pushed
the
feature/azure-image-direct
branch
2 times, most recently
from
e3f074f to
c02507b
Compare
|
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c02507bca3
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
zilianpn
changed the title
|
@luispater 描述可能不准确，是增加 azure openai gpt-image的支持 |
|
OK，我reopen了，回头看一下 |
zilianpn
force-pushed
the
feature/azure-image-direct
branch
3 times, most recently
from
8efeb04 to
e51eb77
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8efeb04681
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
zilianpn
force-pushed
the
feature/azure-image-direct
branch
2 times, most recently
from
8e6e30b to
15319a6
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 15319a6dcb
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
zilianpn
force-pushed
the
feature/azure-image-direct
branch
from
15319a6 to
8d3c82d
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8d3c82ddf9
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
zilianpn
force-pushed
the
feature/azure-image-direct
branch
2 times, most recently
from
a766ab4 to
131724a
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 131724a909
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
luispater
left a comment
luispater
left a comment
There was a problem hiding this comment.
Summary
This PR adds a direct execution path for /v1/images/generations and /v1/images/edits to support Azure OpenAI image deployments (e.g. gpt-image-2) where cross-deployment tool routing via the Responses API can fail. It introduces:
CodexExecutoralt handling forimages/generations+images/editsby constructing{base_url}/{images/*}?api-version=...and forwarding JSON.- OpenAI images handlers that bypass the Responses-orchestration path when a codex provider + base_url is configured.
- Auth conductor tweaks to treat
image_generation_user_error(400) as non-retryable and to stop fallback on HTTP 500. - Restricting
gpt-image-2builtin injection to auth entries that actually configure image models + base_url.
Blocking
streamfield handling for direct routing- The direct generations path forwards the original JSON body as-is. If clients send
stream: true(supported by the gateway’s current orchestration path), that field will be forwarded upstream to/images/generationsand may be rejected as an unknown parameter (or change semantics silently). - Same risk applies to the JSON edits direct path (
directImageEditsJSONcopies the original body). - Suggested fix: strip
stream(and any other gateway-only fields) before callingExecuteWithAuthManager, and explicitly define behavior whenstream=true(reject with a clear error, or ignore and return non-stream response).
- The direct generations path forwards the original JSON body as-is. If clients send
Non-blocking
- Fallback policy change on HTTP 500
isServerErrornow prevents fallback on HTTP 500 acrossExecute,ExecuteCount, andExecuteStream. Please confirm this is intended at global scope (and consider whether 502/503/504 should be treated similarly).
- Azure detection
- Stripping
response_formatbased onbaseURLcontainingopenai/deploymentsis a pragmatic heuristic, but brittle; consider an explicit provider/type marker or URL parsing.
- Stripping
- Edits multipart parameter typing
partial_imagesandoutput_compressionare coerced to integers; please confirm these match the upstream contract (otherwise preserve original types).
Test plan (suggested)
- Add a unit/integration test that exercises the direct route and asserts
streamis not sent upstream (and/or thatstream=trueyields a clear error for direct routing). - Manual sanity on an Azure deployment base_url:
/v1/images/generationswith/withoutstream/v1/images/edits(multipart and JSON) with optional fields (size/quality/background/output_format/etc.)
…ge models Route /v1/images/generations to independent upstream model deployments instead of the Responses API orchestration path. This enables Azure OpenAI gpt-image-2 (and future gpt-image models) which require direct deployment URLs and do not support cross-deployment tool routing. Key changes: - Add images/generations and images/edits endpoint routing in CodexExecutor - Direct path gated to codex provider with configured base_url (e.g. Azure deployment) - Treat image_generation_user_error (400) as non-retryable to prevent AuthManager from falling back to the wrong auth entry - Preserve all caller-supplied edit parameters (size, quality, background, etc.)
zilianpn
force-pushed
the
feature/azure-image-direct
branch
from
131724a to
060d4ad
Compare
已修复。在 ImagesGenerations handler（line ~240）和 directImageEditsJSON（line ~1036）中，进入直连路径前剥离 stream 字段。直连模式本身就是非流式的（ExecuteWithAuthManager 是非流式调用），该字段不应传递给上游。
确认这是全局预期行为。500 是上游服务器错误，切换 auth/provider 无法解决。实际遇到的问题：Azure 返回 500 后，fallback 到另一个 base_url 不匹配的 auth，产生了误导性的 400（unknown_parameter: response_format），掩盖了真实的 500。直接返回原始 500 让客户端能准确重试。502/503/504 属于网络/代理层错误，切换 provider 理论上可能有效，后续可根据需要扩展。
已从 strings.Contains(baseURL, "openai/deployments") 改为 isAzureOpenAIURL() 函数，通过解析 URL host 判断是否以 .openai.azure.com 结尾，比字符串子串匹配更准确。
output_compression（百分比）和 partial_images（数量）确实是 int 类型，与上游 Azure /images/edits 契约一致。
感谢建议，后续会补充单元测试验证 stream 字段剥离和直连路径的端到端流程。 |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 060d4ad90b
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| break | ||
| } | ||
| } | ||
| if hasCodexProvider && h.AuthManager.HasAuthWithBaseURL("codex") { |
There was a problem hiding this comment.
The new direct path is enabled whenever HasAuthWithBaseURL("codex") is true, but that check is global and not tied to the auths that can actually serve the requested image model. Fresh evidence: the hasImageModel/base_url filtering added in buildCodexConfigModels only runs for entries with custom models, so default Codex entries can still advertise gpt-image-2 without a base_url; in that mixed setup this branch forces direct routing and executeImagesGeneration returns 401 missing provider baseURL instead of using the prior Responses fallback.
Useful? React with 👍 / 👎.
2 participants
Description
为 Azure OpenAI 图片生成模型（如
gpt-image-2）添加直接路由支持。Problem
网关正常的图片生成路径通过 Responses API 编排：
这在 OpenAI 上可以正常工作（所有模型共享同一端点），但在 Azure 上会失败。因为 Azure 每个模型是独立的部署，拥有不同的 URL，不支持跨部署的工具路由，导致 tool call 返回 404。
Solution
当检测到图片模型有独立的 upstream 配置（provider 已定义）时，绕过 Responses API 编排路径，直接将请求转发到模型自身的
/images/generations或/images/edits端点。Changes
SDK 图片处理器 (
sdk/api/handlers/openai/openai_images_handlers.go)ImagesGenerations、imagesEditsFromMultipart、imagesEditsFromJSON中增加判断：若模型有 provider 配置，走直接路由directImageGeneration方法：通过 AuthManager 执行直接请求，返回标准 OpenAI 格式响应directImageEditsMultipart和directImageEditsJSON方法：处理 edits 场景的 multipart 和 JSON 请求体转换Codex Executor (
internal/runtime/executor/codex_executor.go)executeImagesGeneration方法：/images/generations或/images/edits）插入 baseURL，保留查询参数（如 Azure 的?api-version=）response_format参数（Azure 始终返回 base64，不接受此参数）Auth Conductor (
sdk/cliproxy/auth/conductor.go)isRequestInvalidError中新增image_generation_user_error判断，将 400 用户错误标记为不可重试，防止 AuthManager 错误地回退到其他 auth entry请求详情 (
sdk/api/handlers/handlers.go)gpt-image-2的拦截逻辑：仅在没有 provider 配置时返回错误，有独立 upstream 配置时允许通过 Responses API 编排路径Affected Files
sdk/api/handlers/openai/openai_images_handlers.gosdk/api/handlers/handlers.gointernal/runtime/executor/codex_executor.gosdk/cliproxy/auth/conductor.go