fix(guard): the changed-path classifier cannot fail open (#384)

.github/workflows/ci.yml

@@ -41,50 +41,41 @@ jobs:
       - uses: actions/checkout@v4
         with:
           fetch-depth: 0
+      # Runs FIRST, for the same reason as the other guardrail self-tests: a
+      # classifier whose own tests fail must not be trusted to decide which
+      # required checks run.
+      - name: Changed-path classifier self-test
+        run: tools/classify_changed_paths.py --self-test
+
+      # The allow-list, the fail-closed behaviour and the rationale all live in
+      # tools/classify_changed_paths.py, which proofs.yml calls too — the two
+      # copies of this logic had already drifted (#379's `rivet.yaml` entry
+      # existed here and not there), which is why it is single-sourced now.
+      #
+      # This step is ONE command on purpose. The old body computed a count in a
+      # subshell and let `[ "$n_code" -gt 0 ]` decide; when grep failed for any
+      # reason the count was empty, the test exited 2, bash read that as false
+      # inside `if`, and the step emitted `code=false` and exited 0 — skipping
+      # 10 required checks (#384). The script therefore appends to
+      # $GITHUB_OUTPUT itself rather than returning a value for the shell to
+      # interpolate. Any `$(...)`, `||` or `-gt` reappearing in this step is
+      # that bug coming back.
+      #
+      # The two event values go through `env:` rather than being interpolated
+      # into the command text. Neither is attacker-controlled (`event_name` is a
+      # GitHub enum, `base.sha` a computed 40-hex sha), but `${{ }}` substitutes
+      # into the script *before* bash parses it, while an env var is only ever a
+      # value. Same reason the rest of this repo's workflows do it.
       - id: filter
         name: Classify diff as code vs artifacts/docs
-        run: |
-          set -euo pipefail
-          if [ "${{ github.event_name }}" != "pull_request" ]; then
-            echo "Non-PR event (${{ github.event_name }}) -> full suite."
-            echo "code=true" >> "$GITHUB_OUTPUT"
-            exit 0
-          fi
-          base="${{ github.event.pull_request.base.sha }}"
-          changed=$(git diff --name-only "$base"...HEAD)
-          echo "Changed files:"
-          printf '%s\n' "$changed"
-          if [ -z "$changed" ]; then
-            echo "Empty diff -> full suite (fail-safe)."
-            echo "code=true" >> "$GITHUB_OUTPUT"
-            exit 0
-          fi
-          # Non-code allow-list: rivet artifacts, STPA files, docs, markdown.
-          # Any path outside it makes the WHOLE diff "code" — a PR that touches
-          # both artifacts and Rust must still run the full suite.
-          # Counted explicitly rather than relying on `grep -q` exit semantics,
-          # so the gate's decision is readable in the job log.
-          #
-          # `rivet.yaml` is on the list (#379) because it is a rivet artifact by
-          # every other statement in this file — the header above says "rivet
-          # artifacts", and the rivet-validate job says it catches regressions
-          # "in artifacts/, safety/stpa/, and rivet.yaml" — but it lives at the
-          # repo root, so `artifacts/.*` never matched it and a rivet.yaml-only
-          # PR ran the entire heavy suite. Safe to skip those jobs for it:
-          # nothing that compiles reads the file (no .rs, no build script, no
-          # Cargo.toml references it; only AGENTS.md, docs/, this workflow, and
-          # artifacts themselves), and it stays gated regardless because
-          # rivet-validate has no `changes` gate at all.
-          allow='^(artifacts/.*\.ya?ml|safety/.*|docs/.*|rivet\.yaml|[^/]*\.md|.*/.*\.md)$'
-          n_code=$(printf '%s\n' "$changed" | grep -cvE "$allow" || true)
-          echo "code-ish files in diff: $n_code"
-          if [ "$n_code" -gt 0 ]; then
-            echo "-> code change: running the full suite."
-            echo "code=true" >> "$GITHUB_OUTPUT"
-          else
-            echo "-> artifacts/docs only: skipping the heavy jobs."
-            echo "code=false" >> "$GITHUB_OUTPUT"
-          fi
+        env:
+          EVENT_NAME: ${{ github.event_name }}
+          BASE_SHA: ${{ github.event.pull_request.base.sha }}
+        run: >-
+          tools/classify_changed_paths.py
+          --event-name "$EVENT_NAME"
+          --base-sha "$BASE_SHA"
+          --github-output "$GITHUB_OUTPUT"
 
   # ── Fast checks ───────────────────────────────────────────────────────
   # fmt / audit / deny / rivet-validate deliberately have NO `changes` gate:

.github/workflows/proofs.yml

@@ -42,26 +42,26 @@ jobs:
       - uses: actions/checkout@v4
         with:
           fetch-depth: 0
+      # This job gates the required `Lean proof typecheck (lake build)` context,
+      # so it is the 11th check the #384 fail-open could skip. It was a verbatim
+      # clone of ci.yml's classifier and had already drifted from it — ci.yml
+      # gained `rivet.yaml` in #379 and this copy never did. Both now call the
+      # same script; see tools/classify_changed_paths.py for the allow-list and
+      # the fail-closed rationale, including why picking up the `rivet.yaml`
+      # entry here is a deliberate (and safe) loosening.
+      - name: Changed-path classifier self-test
+        run: tools/classify_changed_paths.py --self-test
+
       - id: filter
         name: Classify diff as code vs artifacts/docs
-        run: |
-          set -euo pipefail
-          if [ "${{ github.event_name }}" != "pull_request" ]; then
-            echo "code=true" >> "$GITHUB_OUTPUT"; exit 0
-          fi
-          changed=$(git diff --name-only "${{ github.event.pull_request.base.sha }}"...HEAD)
-          printf '%s\n' "$changed"
-          if [ -z "$changed" ]; then
-            echo "code=true" >> "$GITHUB_OUTPUT"; exit 0
-          fi
-          allow='^(artifacts/.*\.ya?ml|safety/.*|docs/.*|[^/]*\.md|.*/.*\.md)$'
-          n_code=$(printf '%s\n' "$changed" | grep -cvE "$allow" || true)
-          echo "code-ish files in diff: $n_code"
-          if [ "$n_code" -gt 0 ]; then
-            echo "code=true" >> "$GITHUB_OUTPUT"
-          else
-            echo "code=false" >> "$GITHUB_OUTPUT"
-          fi
+        env:
+          EVENT_NAME: ${{ github.event_name }}
+          BASE_SHA: ${{ github.event.pull_request.base.sha }}
+        run: >-
+          tools/classify_changed_paths.py
+          --event-name "$EVENT_NAME"
+          --base-sha "$BASE_SHA"
+          --github-output "$GITHUB_OUTPUT"
 
   lean:
     name: Lean proof typecheck (lake build)

artifacts/verification.yaml

@@ -4293,6 +4293,72 @@ artifacts:
     release: v0.36.0
     tags: [process, guardrail, ci, tooling]
 
+  - id: TEST-GUARD-CHANGED-PATHS
+    type: feature
+    title: The changed-path classifier cannot answer "no code" by failing
+    description: >
+      Verifies REQ-GUARD-GATE-EVIDENCE-002 (e). Closes #384.
+
+      `tools/classify_changed_paths.py --self-test` runs 25 cases and is the
+      FIRST step of the `changes` job in both `ci.yml` and `proofs.yml` — a
+      classifier whose own tests fail must not be trusted to decide which
+      required checks run.
+
+      Both original failure modes are regression cases, at the unit level and
+      end-to-end over a real `git diff`:
+
+      MECHANISM A — the allow-list was scoped by PREFIX. `safety/.*` and
+      `docs/.*` admitted any file under those trees, `.rs` and `.sh` included.
+      Every entry is now type-scoped, and `docs/research/forge-replication/`
+      (which already holds `run_trial.sh`, `tasks.json`, `descriptor.sexp` and
+      `descriptor.aadl`) is classified as code.
+
+      MECHANISM B — the error path failed OPEN. A `grep` failure left the count
+      empty, `[ "" -gt 0 ]` exited 2, `set -e` is inert inside an `if`, so bash
+      read that as false and emitted `code=false` with the step exiting 0. Five
+      induced failures — bad base sha, empty base sha, unwritable output,
+      missing arguments — now each assert non-zero AND that no verdict was
+      written.
+
+      The case that makes it a detector rather than an assertion is the
+      PROPERTY, not the cases: for every directory prefix in the allow-list,
+      no pattern may match `<prefix>/evil.rs`, `.sh`, `.toml`, `Makefile`. An
+      editor who re-broadens an entry back to prefix scope fails without anyone
+      having remembered to add a case for them.
+
+      MUTATION-TESTED 2026-08-06, 5 mutants, all caught, each by the test that
+      should catch it: unknown→non-code, `emit()` ignoring its argument,
+      restoring the #384 exception handler, re-broadening `safety/`, re-adding
+      `docs/.*`. The harness asserts each substitution changed the file before
+      drawing a conclusion — a mutation that silently fails to apply reports as
+      "not caught" and is worthless.
+
+      MEASURED on real history: commit 7db7f0e6 (two artifact YAMLs) still
+      emits `code=false`, so the fix tightened the classifier without making it
+      uselessly always-true; commit c8c0450e (8 files, 6 code) emits
+      `code=true`.
+
+      The script owns `$GITHUB_OUTPUT` and appends to it directly. That is its
+      only emission mode on purpose: `echo "code=$(script ...)"` would put
+      mechanism B back in the glue verbatim — script dies, `echo` succeeds,
+      `code=` lands empty, every `== 'true'` dependent skips, exit 0.
+
+      SAFETY ARGUMENT, and its dependency: exiting non-zero is safe only
+      because `Detect changed paths` and `Detect changed paths (proofs)` are
+      themselves required contexts (read from the branch-protection API, not
+      assumed) — a failing `changes` job otherwise skips its dependents, and
+      `skipped` satisfies a required context. If either is ever dropped from
+      the required set this design becomes unsafe; REQ-GUARD-GATE-EVIDENCE-001
+      and `check_required_contexts.py` are what keep that from happening
+      quietly.
+
+      NOT claimed: that skipping is correct for any particular file. This only
+      ensures a skip is caused by a match against a declared, type-scoped
+      pattern and never by a crash.
+    status: implemented
+    release: v0.36.0
+    tags: [process, guardrail, ci, tooling]
+
   - id: TEST-GUARD-RELEASE-PLANE
     type: feature
     title: Release-plane guardrail is proven able to fail before it is allowed to judge