Skip to content

🐛 fix(ci): a skipped coverage upload failed the gate on every fork PR - #815

Merged
nstarman merged 1 commit into
GalacticDynamics:mainfrom
nstarman:claude/fix-coverage-gate
Sep 4, 2026
Merged

🐛 fix(ci): a skipped coverage upload failed the gate on every fork PR#815
nstarman merged 1 commit into
GalacticDynamics:mainfrom
nstarman:claude/fix-coverage-gate

Conversation

@nstarman

@nstarman nstarman commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

CI Pass has been failing on every pull request from a fork, with all seven real jobs green:

📝 upload-coverage-python → ⬜ skipped [required to succeed]

Why

upload-coverage-python needs code-quality: write, which a fork's token does not carry, so it is correctly gated:

if: ${{ ... && (github.event_name != 'pull_request'
                || github.event.pull_request.head.repo.full_name == github.repository) }}

But it is listed unconditionally in the gate's needs, and alls-green counts a skipped required job as a failure. The two are individually right and wrong together.

Nothing was wrong with the coverage job or with the PRs. #812 merged showing FAILURE on a run where format, smoke, tests_linux, tests_nonlinux, tests_subprocess, check_oldest and docs all passed, and #813 is sitting on the same red now. That is the real cost here: a gate that is always red stops carrying information about the runs that do matter, and a genuine failure would look identical.

The fix

allowed-skips: upload-coverage-python — one line, and it excuses the skip and nothing else.

The job stays in needs, so on a same-repo push, where it does run, a failure still fails the gate. allowed-failures was the wrong knob for exactly that reason: it would have excused a genuine upload failure too. Verified against the action's action.yml at the pinned SHA rather than from memory — both inputs exist and the semantics differ as described.

The other seven required jobs carry no if:, so upload-coverage-python is the only one that can skip; this is the whole fix, not one instance of a pattern.

Verification

prek run --all-files clean, including the workflow-schema hook. Parsed the result to confirm the gate still requires the job while excusing only its skip, and that no allowed-failures crept in. The behaviour this changes is only observable in CI: this PR is itself a fork PR, so its own CI Pass is the test — red before, green after.

🤖 Generated with Claude Code

`upload-coverage-python` does not run for a pull request from a fork: it
needs `code-quality: write`, which a fork's token does not carry, so its
`if:` gates on `head.repo.full_name == github.repository`. But it is
listed unconditionally in the `CI Pass` gate's `needs`, and `alls-green`
counts a skipped required job as a failure.

So every fork PR went red with all seven real jobs green -- GalacticDynamics#812 merged
that way, and GalacticDynamics#813 is sitting on the same red now. A permanently failing
gate is worse than no gate: it stops carrying information about the runs
that do matter.

`allowed-skips` excuses the skip and nothing else. The job stays in
`needs`, so on a same-repo push, where it does run, a failure still fails
the gate. `allowed-failures` would have been the wrong knob -- it would
have excused a genuine upload failure too.

The other seven required jobs carry no `if:`, so this is the only one
that can skip.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings September 4, 2026 10:12
@nstarman
nstarman requested a review from a team as a code owner September 4, 2026 10:12
@github-actions github-actions Bot added 👷 Add / update CI build system Add or update CI build system. 🐛 Fix a bug Fix a bug. labels Sep 4, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The change is narrowly scoped, matches the stated CI failure mode, and uses a supported allowed-skips input on the pinned alls-green action version.

Pull request overview

Fixes the CI Pass gate incorrectly failing on fork pull requests when upload-coverage-python is intentionally skipped due to missing code-quality: write permissions on fork tokens.

Changes:

  • Configures the re-actors/alls-green gate to treat upload-coverage-python as an allowed skip (while still failing the gate if that job runs and fails on same-repo PRs/pushes).
File summaries
File Description
.github/workflows/ci.yml Allows the gate to accept a skipped upload-coverage-python job, preventing false-red CI on fork PRs while keeping same-repo enforcement intact.
Review details
  • Files reviewed: 1/1 changed files
  • Comments generated: 0
  • Review effort level: Lite

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@codecov

codecov Bot commented Sep 4, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.91%. Comparing base (66d7489) to head (514aaed).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #815   +/-   ##
=======================================
  Coverage   96.91%   96.91%           
=======================================
  Files         272      272           
  Lines        9433     9433           
=======================================
  Hits         9142     9142           
  Misses        291      291           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@nstarman nstarman added this to the v0.24.0 milestone Sep 4, 2026
@nstarman
nstarman merged commit e8748cb into GalacticDynamics:main Sep 4, 2026
21 checks passed
@nstarman
nstarman deleted the claude/fix-coverage-gate branch September 4, 2026 10:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

👷 Add / update CI build system Add or update CI build system. 🐛 Fix a bug Fix a bug.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants