Master CI gatekeeper: a green PR does not predict a green master

[pftg]

Highest priority. Master CI was red for six consecutive commits while every PR showed green, and a release was nearly cut on it.

Why PR-green cannot predict master-green

.github/workflows/test.yml:104 — the full Test Ruby & Rails matrix is deliberately kept off pull requests:

# Cost-intentional: full matrix stays off PRs by default (free-tier
# Actions minutes). Runs on master pushes, manual dispatch, the weekly
# scheduled drift check, and PRs opted in via the 'full-ci' label.
if: >
  github.ref == 'refs/heads/master' ||
  github.event_name == 'workflow_dispatch' ||
  github.event_name == 'schedule' ||
  contains(github.event.pull_request.labels.*.name, 'full-ci')

That trade-off is reasonable. The consequence is not obvious: the job that breaks master is a job that never ran on the PR, so PR checks are green by construction and every merge is a blind push.

What actually broke (tracked separately)

Subprocess probes chdir into a temp directory — correct, and the documented rule — while inheriting the workflow's relative BUNDLE_GEMFILE: gemfiles/rails81_gems.rb (test.yml:30), which then fails to resolve:

/tmp/d20260824-2715-vtghif/gemfiles/rails81_gems.rb not found (Bundler::GemfileNotFound)

Invisible locally, because bundle exec normalises BUNDLE_GEMFILE to an absolute path before spawning. The documented rule is chdir out and scrub RUBYOPT/BUNDLE_GEMFILE — the chdir landed, the scrub did not. Fix in flight on fix/probe-env-under-ci.

The gatekeeper

• Process rule recorded in the loop protocol: after every merge, check gh run list --branch master --limit 1. A red master blocks the next pull. cancelled is not a pass — it means a later push superseded the run.
• Live monitor watching master's latest run, reporting every terminal conclusion (success, failure, cancelled) rather than only failures — silence must not look like success.
• Label rule: any PR touching subprocess probes, env handling, gemfiles, or CI itself gets the full-ci label before merge, so the matrix runs where it can still block. Document this in CONTRIBUTING.md next to the label.
• Decide whether full-ci should be automatic for paths that have historically broken master — test/**, gemfiles/**, .github/**. A paths-based label bot, or simply making the matrix run on PRs that touch those paths. Costs minutes; buys the guarantee that PR-green means something.
• Branch protection should require the master-push matrix once its check names settle (they change with 2.1: delete the legacy namespace trees and the driver abstraction #249, 2 dimensions → 1). Until then the rule above is the only gate.

The general lesson, for the record

A CI configuration that runs different jobs on PRs than on master is a configuration where green means two different things depending on where you read it. That is defensible for cost, but it must be stated where people merge, not only in a comment inside the workflow.

[pftg]

Status: all five boxes resolved. One of them was decided against the direction this issue proposed.

✅ Process rule — recorded, and it is what actually worked

It caught the red master today, not the monitor. One correction to the command written above: it must be

gh run list --branch master --workflow Test --limit 1

Without --workflow Test you get whichever workflow ran last — it reported a Dependabot success while Test was failing on the same commit, and that false green nearly unblocked a release. In CONTRIBUTING.md.

✅ Live monitor — built, and it lied once before it worked

Rebuilt to emit on arm plus a heartbeat, so silence means dead rather than healthy, and to report cancelled explicitly. Worth recording that the monitor is a convenience, not the gate: the process rule above is the gate.

✅ Label rule — documented in CONTRIBUTING.md

With a cost table for each trigger, and guidance on when to add full-ci by hand: anything conditional on defined? / respond_to? / RUBY_VERSION, anything touching subprocess or env handling, anything you would be surprised to see break on JRuby.

❌ Automatic full-ci on test/**, gemfiles/**, .github/** — implemented, then deliberately reversed

I shipped this in #292 and it worked: it self-triggered on its own PR, 16 cells, no label. Then it was reversed in #295 on the maintainer's call that PRs should run only the critical jobs.

Recording the reasoning rather than just the outcome, because this issue argued the other way:

• The automatic trigger bought "PR-green means something" at ~130 min per qualifying PR — and nearly every PR in this repo touches test/.
• The cheaper route to the same goal was to make being wrong on master cheap instead of making PRs expensive. The master gate went 130 min → ~46 min by replacing the cross product with a spanning set (ci: one JRuby cell instead of five #295), so a red master is now caught and re-run quickly.
• full-ci survives as a manual opt-in for the version-sensitive cases.

This does reopen the gap this issue is named after, and that is a real trade, not a solved problem. It is stated plainly in CONTRIBUTING.md rather than buried in a workflow comment — which was this issue's actual complaint.

✅ Branch protection — unblocked, because the premise was wrong

This box waited on "once the check names settle." They do not settle. Every cell carries its ruby and gemfile in its name, so the required-checks list breaks whenever the matrix changes shape — it just did (16 → 6) and #249 will change it again.

A protection rule that lists cell names silently stops requiring anything the moment a cell is renamed. That is this issue's own failure mode wearing a different hat.

So #295 adds CI Gate: one job that needs: the others and never changes its name.

• skipped is a pass — the matrix does not run on ordinary PRs by design, and failing on skipped would block every PR.
• failure and cancelled are not — a cancelled cell was killed before it reported and carries no verdict. Treating it as success is precisely how the JRuby lane sat broken for 15 consecutive runs.

Logic self-tested on all three shapes before pushing.

Remaining human action, now a single click: in Settings → Branches, require CI Gate on master. That is the whole thing — no list to maintain, and it keeps working through #249.

Closing once #295 merges.