fix(proxy): contain post-terminal health write failures

[JustYannicc]

Summary

A failing account-health write after a Responses terminal can emit a second response.failed event or abort stream completion. Catch and log ordinary post-terminal health exceptions while preserving the original response, reservation settlement order, and cancellation propagation.

Type of change

• fix: — bug fix (no behavior change beyond the bug)

Linked issue: Fixes #2033

OpenSpec

• This PR includes / updates an OpenSpec change
• This PR touches a codex-faithful path and preserves upstream-equivalent behavior

Change directory: openspec/changes/archive/2026-09-10-fix-post-terminal-health-failures/

Changes

• Use one exception-logging helper at the six post-terminal error-health writes. Existing settlement gates, queued-account deduplication, and HTTP status forwarding stay intact.
• Add a real ASGI /v1/responses regression and four stream-handler cases covering keyed owner continuations and an unkeyed stream with visible text followed by a quota error.

Pre-terminal health writes, settlement failures, and success recording keep their existing behavior. This is independent of unmerged PRs, including #1905.

Test plan

Both database environment variables pointed to the same dedicated disposable SQLite database before app imports and tests.

uv run pytest tests/unit/test_proxy_utils.py \
  -k 'stream and (health or settlement or terminal or owner_rewrite)' -q
# 72 passed
uv run pytest tests/integration/test_proxy_responses.py \
  -k test_v1_responses_health_write_failure_keeps_one_terminal -q
# 1 passed
make lint typecheck
# Passed, including architecture and cancellation checks
pnpm dlx @fission-ai/openspec validate --specs --strict
# 65 passed

The four new stream-handler cases and the public HTTP test fail against pinned main 0f6a31c56 and pass with this patch. The active change passed strict validation before archive; main specs passed again after sync. Independent review found no actionable defects. Hosted CI and maintainer review remain pending.

Screenshots / output

Public request: POST /v1/responses with {"model":"gpt-5.1","instructions":"hi","input":"hello","stream":true}. A synthetic upstream emits text and then raises a quota error; the injected health write raises RuntimeError.

• Before: response.failed/usage_limit_reached, then response.failed/upstream_error.
• After: one response.failed/usage_limit_reached; the health failure is logged with exception information.

Checklist

• Title is in Conventional Commits format.
• Linked the related issue above.
• Added tests covering the change.
• Ran the relevant make subset locally.
• Strict OpenSpec validation and implementation verification pass.
• Simplicity gates reviewed; no settings, defaults, setup steps, README sections, or dashboard changes.
• CHANGELOG is not edited by hand.

[coderabbitai]

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Advanced

Run ID: 47361d9c-af15-4436-bb88-999dfef92f74

📥 Commits

Reviewing files that changed from the base of the PR and between 0f6a31c and 26cfb9e.

📒 Files selected for processing (12)

• app/modules/proxy/_service/streaming/retry.py
• openspec/changes/archive/2026-09-10-fix-post-terminal-health-failures/.openspec.yaml
• openspec/changes/archive/2026-09-10-fix-post-terminal-health-failures/context.md
• openspec/changes/archive/2026-09-10-fix-post-terminal-health-failures/design.md
• openspec/changes/archive/2026-09-10-fix-post-terminal-health-failures/proposal.md
• openspec/changes/archive/2026-09-10-fix-post-terminal-health-failures/specs/responses-api-compat/spec.md
• openspec/changes/archive/2026-09-10-fix-post-terminal-health-failures/tasks.md
• openspec/changes/archive/2026-09-10-fix-post-terminal-health-failures/verification.md
• openspec/specs/responses-api-compat/context.md
• openspec/specs/responses-api-compat/spec.md
• tests/integration/test_proxy_responses.py
• tests/unit/test_proxy_utils.py

Included review availability: Your plan provides up to 8 included reviews per hour; 4 remain after this review.

---

📝 Summary

Summary by CodeRabbit

• Bug Fixes

  • Streaming Responses now remain complete when a post-terminal account health update fails.
  • Prevented duplicate terminal events or interrupted streams in these failure scenarios.
  • Health-write failures are logged while preserving cancellation and settlement behavior.

• Tests

  • Added coverage for keyed and unkeyed requests, multiple upstream error paths, and terminal event handling.

Walkthrough

The retry loop now contains ordinary failures from post-terminal account-health writes. Responses streams preserve the original terminal event and completion. Specifications and regression tests cover keyed, unkeyed, first-event, later-event, and upstream-error paths.

Changes

Post-terminal health failure handling

Layer / File(s)	Summary
Responses completion contract openspec/changes/archive/2026-09-10-fix-post-terminal-health-failures/*, openspec/specs/responses-api-compat/*	The specifications define logging health-write failures without emitting another terminal event or aborting completion. They require settlement before keyed health writes.
Post-terminal health-write containment app/modules/proxy/_service/streaming/retry.py	A local helper catches and logs ordinary health-write exceptions. Six post-terminal call sites use the helper, including the path that preserves http_status.
Single-terminal regression coverage tests/unit/test_proxy_utils.py, tests/integration/test_proxy_responses.py	Tests cover keyed and unkeyed failures, settlement ordering, one terminal event, original error codes, and logged health-write exceptions.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Severity of issue fixed: Medium

Suggested reviewers: soju06

Merge Risk: ⚪ Minimal · up to 26cfb

The streaming change preserves terminal-response completion while containing post-terminal health-write failures; no merge-blocking risk remains.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 10 functions across 2 files. (10 skipped: …	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and concisely describes the primary fix: containing failures from post-terminal health writes in proxy streaming.
Description check	✅ Passed	The description directly explains the duplicate-terminal-event bug, the containment strategy, preserved behavior, linked issue, tests, and validation results.
Linked Issues check	✅ Passed	The implementation satisfies issue #2033 by wrapping all six post-terminal health writes, logging ordinary exceptions, preserving the original terminal response, and maintaining settlement and cancell…
Out of Scope Changes check	✅ Passed	The code, regression tests, OpenSpec updates, and verification documents are directly related to fixing issue #2033. No unrelated production behavior or configuration changes are present.

Full details: Docstring Coverage

Explanation

Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 10 functions across 2 files. (10 skipped: 9 unsupported, 1 too large.)

• Fix all pre-merge checks with AI

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[JustYannicc]

@coderabbitai review

Please review head 26cfb9e8b18b3391a78c46877fe7e6be85f25aed. No submitted CodeRabbit review exists for this PR. The 08:17 UTC response advertised a 31-minute cooldown, which has elapsed. This is one request to check restored review capacity; CI is green.

[coderabbitai]

@JustYannicc I will review the requested head commit and the PR changes.

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[Komzpa]

@codex review

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-09-10T21:24:18.112611Z	26cfb9e	Manual request

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. What shall we delve into next?

Reviewed commit: 26cfb9e8b1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[github-actions]

This pull request has been automatically marked as stale because it has had no activity for 7 days.

It will be closed in 23 more days unless there is new activity.

If this is still relevant, please:

• Rebase or push an update if the branch drifted
• Address pending review feedback if there is any
• Leave a short comment confirming it is still being worked on

Thanks for the contribution 🙏