Skip to content

fix(gateway): preserve system block cache control during rewrite#2949

Open
black-06 wants to merge 2 commits into
Wei-Shaw:mainfrom
black-06:fix_rewrite_system
Open

fix(gateway): preserve system block cache control during rewrite#2949
black-06 wants to merge 2 commits into
Wei-Shaw:mainfrom
black-06:fix_rewrite_system

Conversation

@black-06
Copy link
Copy Markdown

@black-06 black-06 commented Jun 1, 2026

摘要

在 rewriteSystemForNonClaudeCode 时, 不合并为字符串, 而是将原始的 system content 以 block 为单位迁移 (包括 cache_control)

背景

问题

在使用 sub2api 作为 base_url 测试缓存命中时发现, 如果手动控制 system 块缓存, 多次请求缓存无法命中

最小复现请求是:

{
    "model": "claude-opus-4-7",
    "max_tokens": 64,
    "system": [
        { "type": "text", "text": "system prompt 1 ..." },
        { "type": "text", "text": "system prompt 2 ...", "cache_control": { "type": "ephemeral" } }
    ],
    "messages": [
        { "role": "user", "content": "reply with one short sentence." }
    ]
}

sub2api 会将 system 注入到 messages 中,改写影响了对应的 cache_control:

{
    "model": "claude-opus-4-7",
    "max_tokens": 64,
    "messages": [
        {
            "role": "user",
            "content": [
                { "text": "[System Instructions]\nsystem prompt 1 ...\n\nsystem prompt 2 ...", "type": "text" }
            ]
        },
        { "role": "assistant", "content": "Understood. I will follow these instructions." },
        { "role": "user", "content": "Request 1: reply with one short sentence." }
    ],
    "system": ...
}

方案

将 system text block 整个迁移为 user message content block, 保留 cache_control 属性

{
    "model": "claude-opus-4-7",
    "max_tokens": 64,
    "messages": [
        {
            "role": "user",
            "content": [
-                { "text": "[System Instructions]\nsystem prompt 1 ...\n\nsystem prompt 2 ...", "type": "text" }
+                { "text": "[System Instructions]", "type": "text" },
+                { "text": "system prompt 1 ...", "type": "text" },
+                { "text": "system prompt 2 ...", "type": "text", "cache_control": { "type": "ephemeral" } },
            ]
        },
        { "role": "assistant" "content": "Understood. I will follow these instructions." },
        { "role": "user", "content": "Request 1: reply with one short sentence." }
    ],
    "system": ...
}

Keep original system text blocks as separate migrated message content
blocks instead of joining them into one string, and preserve each block's
cache_control metadata when rewriting non-Claude-Code requests.
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 1, 2026

All contributors have signed the CLA. ✅
Posted by the CLA Assistant Lite bot.

@black-06
Copy link
Copy Markdown
Author

black-06 commented Jun 1, 2026

I have read the CLA Document and I hereby sign the CLA

github-actions Bot added a commit that referenced this pull request Jun 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant