Bug: 总供应商数: 0 个的时候一直等到超时结束

[xujiesh0510]

响应时间过长：

0000ms] 会话复用选择供应商

Session ID: 55ecfbe2-495e-4e6f-8a36-673ead0f212c
复用供应商: tiger
供应商配置: 优先级=0 权重=5 成本倍数=1
基于会话缓存（5分钟内）

等待请求结果...

[0000ms] 首次选择供应商

系统状态:
总供应商数: 0 个
已启用 claude 类型: 0 个
健康检查通过: 0 个

✓ 选择: tiger

等待请求结果...

[128494ms] 请求失败（第 1 次尝试）

供应商: tiger
状态码: 524
错误: Provider returned 524:
请求耗时: 128494ms

熔断状态:
当前状态: 关闭（正常）
失败次数: 1/5
距离熔断还有 4 次

错误详情:

<title>bookapi.cc | 524: A timeout occurred</title> [255115ms] 供应商类型全端点超时（524）

供应商: tiger
状态码: 524
错误: Provider returned 524:
请求耗时: 126621ms

错误详情:

<title>bookapi.cc | 524: A timeout occurred</title> 该供应商类型的所有端点均超时，已触发供应商类型临时熔断。

[258740ms] cubence (client_abort)

==========================================================

claude code分析：

---

根因分析

你设置了 30s 流式超时（firstByteTimeoutStreamingMs = 30000），但实际等了 128 秒才超时。这是两条独立代码路径的问题：

路径分叉点（forwarder.ts:741）

send()
├─ shouldUseStreamingHedge() == true → sendStreamingWithHedge() ← 用 thresholdTimer
└─ shouldUseStreamingHedge() == false → 普通重试循环 ← 用 responseTimeout

shouldUseStreamingHedge 的触发条件（forwarder.ts:2951-2958）

(endpointPolicy?.allowRetry ?? true) &&
(endpointPolicy?.allowProviderSwitch ?? true) &&
stream === true &&
firstByteTimeoutStreamingMs > 0 // ← 你设置了 30s，所以走 hedge 路径

Hedge 路径里的 firstByteTimeoutMs 是干什么的（forwarder.ts:3117-3118）

const providerForRequest =
attempt.firstByteTimeoutMs > 0
? { ...attempt.provider, firstByteTimeoutStreamingMs: 0 } // ← 传给 doForward 时清零！
: attempt.provider;

Hedge 路径把 firstByteTimeoutStreamingMs 清零再传给 doForward，这样 doForward 内部的 responseTimeoutMs = 0，即禁用了 abort 超时（forwarder.ts:2334）。

Hedge 路径自己用 thresholdTimer（armAttemptThreshold）来触发"切换备用供应商"，而不是中止当前请求。当没有其他可用供应商时（日志显示：总供应商数: 0，只有 tiger 一个），launchAlternative() 找不到备选，于是
当前的 tiger 请求就不受任何超时约束地一直跑下去，直到上游 Cloudflare 的 524（120s）。

为什么没有备选供应商？

日志里明确写了：
总供应商数: 0 个
已启用 claude 类型: 0 个
只有一个 tiger，launchAlternative 返回 null，Hedge 机制退化为"什么都不做"，但此时 doForward 的 abort 已经被清零了，所以 30s 超时完全失效。

本质上是一个 bug

Hedge 路径的设计意图是：30s 内没有首字节 → 触发切换到备用供应商。但当没有备用供应商时，应该直接 abort 当前请求，而不是让它无限等待。当前代码在 launchAlternative 返回 null 后只是 return，没有 abort 掉
in-flight 的请求。

修复思路

在 launchAlternative() 确认没有备选供应商之后，如果当前还有 in-flight 请求，应该根据 firstByteTimeoutMs 给它们设一个兜底 abort（或者干脆在 shouldUseStreamingHedge 里加个"至少有 2 个供应商"的前置条件）。

最简单的修复在 launchAlternative（forwarder.ts:3080-3086）：

if (!alternativeProvider) {
noMoreProviders = true;
if (attempts.size === 0) {
await finishIfExhausted();
} else {
// 没有备选时，给所有 in-flight 请求补上首字节超时兜底 abort
for (const attempt of Array.from(attempts)) {
if (attempt.firstByteTimeoutMs > 0 && !attempt.settled) {
attempt.responseController?.abort();
}
}
}
return;
}

不过更彻底的修复是在 runAttempt 里：当 noMoreProviders 确认后，如果 thresholdTriggered 但没有 winner，应把当前请求的 abort 恢复。你可以根据实际架构选择合适的修复层次。

[coderabbitai]

🔗 Related PRs

#554 - feat: 添加供应商查询缓存, 改善性能 [merged]
#759 - fix(proxy): 修复请求卡死（AgentPool 驱逐阻塞） [merged]
#762 - fix(proxy): 加固 AgentPool 清理与透传 stats [merged]
#837 - 修复：熔断器禁用后仍拦截供应商 [merged]
#961 - fix: 模型列表端点返回所有可用供应商的模型而非仅第一个 [merged]

---

📝 Issue Planner

_{Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.}

• Create Plan

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!

[github-actions]

Thank you for the detailed bug report and the excellent root cause analysis. Your investigation is accurate — I've verified the code paths you identified.

Analysis:

The bug is confirmed. When a streaming hedge request has firstByteTimeoutStreamingMs > 0 but only one provider is available, the timeout effectively becomes infinite:

1. Hedge path is entered (forwarder.ts:2954-2961): The shouldUseStreamingHedge() check passes because firstByteTimeoutStreamingMs > 0, regardless of how many providers are available.

2. firstByteTimeoutStreamingMs is zeroed for doForward (forwarder.ts:3133-3136): The hedge path intentionally clears firstByteTimeoutStreamingMs before passing the provider to doForward, because the hedge layer manages timeouts via armAttemptThreshold instead.

3. doForward timeout is disabled (forwarder.ts:2334-2364): With firstByteTimeoutStreamingMs = 0, responseTimeoutMs becomes 0, so no abort timer is set. The request runs with no timeout.

4. Threshold timer fires but has no effect (forwarder.ts:3059-3069): When armAttemptThreshold fires after the configured timeout, it calls launchAlternative().

5. launchAlternative finds no alternatives (forwarder.ts:3097-3104): With only one provider, selectAlternative returns null. The code sets noMoreProviders = true and returns — but critically does not abort the in-flight attempt:

   if (\!alternativeProvider) {
     noMoreProviders = true;
     // No alternative providers available — let in-flight attempt(s) continue.
     if (attempts.size === 0) {
       await finishIfExhausted();
     }
     return;  // <-- in-flight request continues with no timeout
   }
   

The comment on line 3099 ("let in-flight attempt(s) continue") reveals this is intentional design for multi-provider scenarios where it's better to wait for an in-flight request than to fail. But when there's only one provider, this means the request runs indefinitely until the upstream times out (Cloudflare 524 at ~120s in your case).

Affected code:

• src/app/v1/_lib/proxy/forwarder.ts (lines 3097-3104) — launchAlternative doesn't abort in-flight attempts when no alternative exists
• src/app/v1/_lib/proxy/forwarder.ts (lines 3133-3135) — firstByteTimeoutStreamingMs zeroed before doForward
• src/app/v1/_lib/proxy/forwarder.ts (lines 2954-2961) — shouldUseStreamingHedge has no minimum provider count check

Your suggested fix directions are sound. Either:

1. In launchAlternative (line 3097-3104): when no alternative is found and noMoreProviders becomes true, apply a fallback abort timeout to in-flight attempts that have already exceeded their firstByteTimeoutMs.
2. In shouldUseStreamingHedge: add a precondition that at least 2 eligible providers exist, so single-provider setups fall through to the regular retry loop which respects firstByteTimeoutStreamingMs directly.

---

Automated response from Claude AI