Repository navigation
[adopter] The markdown layer has no compiler: governance artifacts drift from the tree because nothing mechanically rejects them #419
Description
Adopter: Sentinel (adoption discussion)
Finding type: Pattern candidate — recurring anti-pattern or discipline worth documenting
Charter reference: CHARTER-61 (Sentinel), external audit rounds 1–2
N-context: Recurrence within one adopter: a defect class documented as prose in audit round 1 was found again, in a new form, in round 2. Rhymes with #304 (mocks encode the assumed contract) and #306 (verification fidelity).
Summary
A two-round external audit of a Sentinel Charter (multi-model auditors, evidence-cited reports) produced a diagnosis that generalizes: the code has mechanical verifiers; the governance markdown does not. Compilation, vet, tests, SAST and DI wiring all reject bad code before it lands. AILOGs, the follow-ups registry, Charters and commit messages are only checked by prose checklists that an agent must remember to run. The residual defects clustered exactly in the unguarded seams — never in the code's logic, always in the artifacts that claim to verify or describe it.
Evidence (three cases from the same audit round)
1. A phantom AILOG reference in a commit message. A remediation commit cites AILOG-2026-08-12-002. The format is right, the date is plausible, the sequence number is plausible — and the file does not exist in the tree. Nothing in the toolchain resolves id-shaped references, so a confabulated citation lands silently. This failure mode is agent-specific in kind: a human forgets to write a document; an agent invents its exact identifier with full confidence.
2. Follow-ups that contradict the tree. Two open follow-ups (FU-377, FU-379) assert that a background job "has no production caller" — while the application entrypoint wires it. The registry tracks what AILOGs say, not what the code does, and nothing re-derives the claim. The drift is invisible until a human happens to cross-check.
3. A root-cause lesson that stayed prose, and recurred. Round 1 of the audit named a root cause: test doubles encode what the domain assumes, not what the real component does — producing guards that cannot fail. It was written into an AILOG and into the round-2 audit prompt itself. Round 2 found the same class in a new form: an integration test embedding a hand-copied copy of the SQL that the query layer generates, so the test stays green even if the generated query changes. The lesson existed; the gate did not.
The adopter's own doctrine already states the principle — a rule you must remember has the same defect as the bug it watches — and its hard guards (a blocking PR-voice hook, an agent-rules sync check, a merge driver for the follow-ups registry) are all reactive, each born from one measured failure. Guard coverage grows by accident.
Why this is upstream-shaped
The framework owns the id namespaces (AILOG-, FU-, CHARTER-*), so referential integrity over them can only be enforced generically by the framework. And the CLI already has every surface the remediation needs:
straymark validate --include-chartersalready performs referential integrity — scoped to Charter frontmatter (originating_ailogs must exist). The concept exists; its scope is narrow.straymark validate --stagedis already designed for hook integration.straymark analyze declared-vs-wiredalready proves the right architecture: engine upstream, rules as adopter config.straymark followupsowns the registry artifact whose claims drifted in case 2.
Proposed upstream changes (three, ordered by value/effort)
P1 — Generalized reference resolution: "name resolution" for the markdown layer. Extend validate's referential integrity beyond Charter frontmatter: AILOG related: fields must resolve; id-shaped tokens cited anywhere in .straymark/ must resolve; and a new mode along the lines of validate --commit-msg <file> should extract id-shaped tokens from a commit message and fail when any does not resolve — designed for commit-msg hooks the way --staged is designed for pre-commit. The id formats are framework-owned and unambiguous, so precision is total and the check can be blocking from day one. This eliminates the phantom-reference class (evidence case 1).
P2 — A guard-closure field in remediation AILOGs. When an AILOG records the remediation of audit findings, each identified root cause must close with either a named, executable gate (guard:) or an explicit, measured justification for not gating it (unguardable:) — modeled on the existing escape-hatch conventions that require a non-generic rationale. This converts "lesson as prose" into "lesson as gate, or documented exception" (evidence case 3). The audit-review skill could require the field when consolidating remediation rounds.
P3 — followups verify. A mode that re-derives code claims embedded in open follow-ups. Language-agnostic form: a follow-up that asserts something about a symbol or path names it in backticks; verify greps the tree and flags entries whose claim no longer holds — a referenced path gone, or a "no caller" claim that now has a caller. This would have flagged case 2 without knowing anything about the adopter's language. Warn-first, since claim phrasing is semi-structured; the backtick convention makes it precise over time.
What stays adopter-side
Project-semantics checks — e.g. "integration tests must execute the generated query rather than a hand-copied copy" — remain adopter guards or declared-vs-wired profiles. The division principle is the one declared-vs-wired already implements: engine upstream, rules as config downstream.
Design constraints for any implementation
- High-precision checks block; heuristic checks warn — follow the declared-vs-wired precedent of warning mode first, blocking flip later.
- All checks must live in the grep/parse tier — no build, no test execution — or the hook that hosts them becomes too slow to tolerate.
- Every new gate should declare the defect class it covers, so guard coverage grows deliberately rather than by accident.
Relation to existing issues
- Verification fidelity: a 'verified/resolved' claim should record *what reality* it was checked against (proxy vs the condition that matters) #306 asks against what reality a "verified" claim was checked; this asks whether the claim is mechanically checked at all. Complementary layers.
- Cross-spec decision propagation: a post-MVP backend decision silently diverged a later consumer spec's contract — the system view should link them #304 covers mocks encoding the assumed contract — the test-double side of evidence case 3.
- Audit-prompt: add cross-boundary contract verification (client-consumes-server in monorepo) #303 is adjacent (what to verify across boundaries).
Filed from real adopter experience (Sentinel, CHARTER-61 external audit rounds 1–2, multi-model auditors). The load-bearing lesson: an agent converges to whatever a machine checks — artifacts without a mechanical verifier drift, no matter how disciplined the prose around them.
Activity
feat(cli): cli-3.46.0 — generalized reference resolution + validate -…
feat(cli): cli-3.47.0 — followups verify --claims (#419) (#421)
feat(cli+framework): fw-4.44.0 / cli-3.48.0 — guard-closure in remedi…
chore(charter): close CHARTER-02 — mechanical verifiers for the markd…
montfort commentedon Aug 14, 2026
Shipped via CHARTER-02 (three PRs, all merged):
IdIndex(documents + FU registry + charters) behind REF-001 (related:now an Error), new REF-003 (body scan for unresolvable id-shaped tokens, warn-first), and blockingvalidate --commit-msg <FILE>(COMMIT-REF-001) for commit-msg hooks. Addresses case 1 (phantom AILOG citation).followups verify --claims— batch re-derivation of registry code claims (CLAIM-PATH-GONE / CLAIM-SYMBOL-GONE / CLAIM-STALE-DEAD with a ≥2-file precision guard), grep tier, warn-first. Addresses case 2 ("no callers" claims that decayed).guard_closure:field on remediation AILOGs (signature:trigger:) — one item per closed finding, exactly one ofguard:/unguardable:; GUARD-001 (warn-first) flags missing/ambiguous/generic closures;charter amendscaffolds the block; the audit-review skill feeds it upstream. Addresses case 3 (prose-only remediations that recur).Deliberately not done (by design constraint "high-precision checks block, heuristic checks warn"): REF-003, GUARD-001 and CLAIM-* stay warn-first — their flip to Error waits for a measured adopter baseline (FU-007 tracks the REF-003 side). All gates declare their defect class; everything runs at grep/parse tier, no AST.
Charter closed with full drift reconciliation (zero scope-expansion across the three PR ranges): https://github.com/StrangeDaysTech/straymark/blob/main/.straymark/charters/02-mechanical-verifiers-for-the-markdown-layer-419.md