Skip to content

feat(guard): a sorry cannot exempt itself by writing a comment (#385) - #396

Merged
avrabe merged 2 commits into
mainfrom
feat/v036-lean-sorry-floor
Aug 7, 2026
Merged

feat(guard): a sorry cannot exempt itself by writing a comment (#385)#396
avrabe merged 2 commits into
mainfrom
feat/v036-lean-sorry-floor

Conversation

@avrabe

@avrabe avrabe commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

REQ-GUARD-GATE-EVIDENCE-002 (f) — v0.36.0 bug-fix track. Written during the GitHub Actions outage, so CI here will be red for infrastructure reasons until the incident clears; see #395 for the same situation.

The defect

Fail on sorry (post-build gate) is part of the required Lean proof typecheck (lake build) context. It permitted every sorry in the tree while printing:

No sorrys in proofs/Proofs/ — gate green.

It excluded any sorry with a same-line -- TODO, and all twelve are sorry -- TODO(v1.0.0). Twelve in, zero out, the if never fires. The gate was satisfied by exactly the thing it exists to prevent.

How it drifted — the part worth keeping

The step's own comment said:

"The five tracked sorrys do NOT carry a same-line comment; the TODO sits on the prior line, so this gate flags all five."

That was true when written. The exclusion was a deliberate no-op against a form nobody used. The sorries then grew 5 → 12 and moved their TODO onto the same line — and the no-op silently became universal. Nobody edited the guard; the code drifted out from under its comment.

So the defect isn't a careless filter. It's that a self-exemption keyed on a pattern the exempted code itself controls cannot hold — the proof author writes both the sorry and the -- TODO that excuses it, in one commit, and no review step sees the exemption grow.

The replacement

A ratchet, same shape as the mutants gate. No comment-based exemption at all — a -- TODO is a note to humans, not a permit — and the floor lives in proofs.yml, not in the proofs.

condition behaviour
count > floor FAIL — new sorries cannot be admitted by writing a comment
count < floor PASS, loudly saying to lower the floor. Improving must never fail, or the ratchet gets walked back instead of tightened
empty scan exit 2 — zero sorries because no .lean was read is the error-path-yields-ideal-reading shape this requirement names

Count plus per-file, per-line breakdown prints on every path including success, so "12 permitted" is visible rather than inferred from silence.

Measured

On ae27ff1: 11 .lean files scanned, 12 sorries (MinPlus.lean 4, MinPlusPwa.lean 8). Floor set to 12 — green and honest rather than green and blind. Verified it can go red: at --max-sorries 11 it exits 1 naming the breach.

8-case self-test runs before the gate judges anything. The discriminating pair is the regression itself: sorry -- TODO counts and breaches a floor of 0, and the same file passes at a floor of 1 — the comment is signal for neither verdict. A checker still honouring the exemption passes the first case and cannot fail it. Also pinned: sorryAx and "sorry" in prose are not bare sorries.

Not claimed

That the 12 remaining sorries are acceptable. The floor records how many exist, not that they're fine — discharging them is REQ-PROOF-NC-MINPLUS-001 (v0.38.0). This only stops the number rising unnoticed, which is exactly what it did between 5 and 12.

Refs #385

🤖 Generated with Claude Code

REQ-GUARD-GATE-EVIDENCE-002 (f).

`Fail on sorry (post-build gate)` is part of the REQUIRED `Lean proof typecheck
(lake build)` context. It permitted every sorry in the tree while printing

    No sorrys in proofs/Proofs/ — gate green.

Its pipeline excluded any sorry carrying a same-line `-- TODO`, and all twelve
are `sorry -- TODO(v1.0.0)`. Twelve in, zero out, the `if` never fires. The gate
was satisfied by exactly the thing it exists to prevent.

HOW IT DRIFTED, which matters more than the filter.

The step's own comment said:

    # The five tracked sorrys do NOT carry a same-line comment; the
    # TODO sits on the prior line, so this gate flags all five.

That was TRUE when written. The exclusion was a deliberate no-op — a defensive
filter against a form nobody used. The sorries then grew 5 -> 12 and moved their
TODO onto the same line, and the no-op silently became universal. Nobody edited
the guard. The code drifted out from under its comment.

So the defect is not a careless filter. It is that A SELF-EXEMPTION KEYED ON A
PATTERN THE EXEMPTED CODE ITSELF CONTROLS CANNOT HOLD: the proof author writes
both the `sorry` and the `-- TODO` that excuses it, in the same commit, and no
review step sees the exemption grow. An exemption has to be declared somewhere
the exempted party does not edit.

THE REPLACEMENT is a ratchet, the same shape as the mutants gate. Count every
bare sorry with NO comment-based exemption — a `-- TODO` is a note to humans,
not a permit — and compare against `--max-sorries`, declared in proofs.yml
rather than in the proofs.

  count > floor  FAIL. New sorries cannot be admitted by writing a comment.
  count < floor  PASS, and say loudly to lower the floor. Improving must never
                 fail, or the ratchet gets walked back instead of tightened.
  empty scan     exit 2. Zero sorries because no .lean file was read is the
                 error-path-yields-ideal-reading shape this requirement names.

The count and a per-file, per-line breakdown print on EVERY path including
success, so "12 permitted" is visible rather than inferred from silence.

MEASURED on ae27ff1: 11 .lean files scanned, 12 sorries (MinPlus.lean 4,
MinPlusPwa.lean 8). Floor set to 12 — green and honest rather than green and
blind. Verified it can go red: at `--max-sorries 11` it exits 1 naming the
breach.

Self-test, 8 cases, runs before the gate judges anything. The discriminating
pair is the regression: `sorry -- TODO` COUNTS and breaches a floor of 0, and
the SAME file passes at a floor of 1 — the comment is signal for neither
verdict. A checker still honouring the exemption passes the first case and
cannot fail it. Also pinned: `sorryAx` and the word "sorry" in prose are not
bare sorries, so the count cannot be inflated by unrelated text.

NOT claimed: that the 12 remaining sorries are acceptable. The floor records how
many exist, not that they are fine. Discharging them is REQ-PROOF-NC-MINPLUS-001
(v0.38.0). This only stops the number rising unnoticed — which is precisely what
it did between 5 and 12.

Refs #385
@github-actions

github-actions Bot commented Aug 6, 2026

Copy link
Copy Markdown

Rivet verification gate

20/20 passed

count
Passed 20
Failed 0
Skipped (no steps) 0

Filter: (and (= type "feature") (or (has-tag "v093") (has-tag "v0100")))

Failed artifacts

(none)

Updated automatically by tools/post_verification_comment.py. Source of truth: artifacts/verification.yaml.

@codecov

codecov Bot commented Aug 6, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

#395 (obligations g+h) and this branch (f) both appended a TEST-GUARD-*
artifact at the same insertion point in artifacts/verification.yaml, so the
three-way merge conflicted on adjacency alone. Both sides kept; nothing was
edited or dropped.

Verified on the MERGED tree, not on either parent: 168 artifacts, no
duplicate ids, all three new entries present, check_release_plane /
check_human_scoped / check_required_contexts green, and the four guard
self-tests (lean_sorries 8, clippy_workspaces 3, fuzz_targets 7,
fmt_workspaces) all pass together. Live gates on the merged tree:
"All 3 declared fuzz targets are run" and "12 sorries, at or under the
declared floor of 12".

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@avrabe
avrabe merged commit 184a93a into main Aug 7, 2026
21 checks passed
@avrabe
avrabe deleted the feat/v036-lean-sorry-floor branch August 7, 2026 02:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant