Skip to content

ABB: role becomes a closed enum, and the vendor control gets teeth - #225

Merged
mdheller merged 1 commit into
mainfrom
feat/abb-role-enum-and-teeth
Jul 30, 2026
Merged

ABB: role becomes a closed enum, and the vendor control gets teeth#225
mdheller merged 1 commit into
mainfrom
feat/abb-role-enum-and-teeth

Conversation

@mdheller

Copy link
Copy Markdown
Contributor

#223 shipped three instances of the defect class it was written to close. Found by turning today's Apple-container lens back on our own spec.

1. The vendor-role control never looked at a document

It built one and then never used it:

e = copy.deepcopy(example)
e["role"] = v          # <- assigned, never passed to a validator
if _VENDOR_ROLES_FORBIDDEN.match(v):   # matches the STRING, not the document

So the regex was only ever tested against its own six fixtures. With role: "postgres" in the shipped example, the validator exited 0 and printed OK 6 vendor-named roles caught by the deny list. A control that reports catching what it has not looked at is worse than an absent one — it produces evidence.

2. A deny list cannot prevent the failure that compounds

role was {type: string, minLength: 1}. Vendor names are the obvious slip; the one that actually accumulates is the same role spelled several ways. Measured on macOS 26.3.1 — 627 sandbox containers, extension-point names including:

spelling count
extension 45
DiagnosticExtension 8
diagnosticextension 5
diagnostic 3
Extension 2

Fifteen years of a freeform role field with no enforced grammar. Past that point the name cannot group anything and a side-channel record has to carry the contract instead. An allowlist forecloses both failures at once, and forecloses them in the schema — the artefact every consumer already validates against, unlike a validator's private regex.

3. conformanceTest pointed at a file that does not exist

conformance/abb-database-vectors.json is not in the repo. format: uri-reference is a syntax check and resolves nothing, so the dangling pointer validated clean — and read as "held to shared vectors" to every consumer that did not go looking.

What changed

  • role is a closed enum over the four roles actually in use (prophet-mesh specs/abb-catalog.yaml + the example), never roles we imagine wanting. Adding one is a spec PR — the reviewed gate whose absence produced the five spellings above.
  • The deny list is deleted. Its cases now go to the schema as documents: 8 vendor names, 6 case/whitespace variants, 1 invented role — all rejected.
  • INVARIANT 1a mutation-tests the enum. Without it, the cases above keep passing if role reverts to a freeform string while some unrelated constraint does the rejecting. Drop the enum ⇒ postgres accepted; restore ⇒ rejected.
  • INVARIANT 3 resolves the pointer, with a negative control proving a dangling one is caught. The example's dangling pointer is removed, not backfilled — absence is documented as meaningful, and inventing vectors no implementation runs would be a new paper control.

Verified by probe, not by exit code

document before after
`role: postgres" rc=0, printed OK rc=1
role: database (case variant) rc=0 rc=1
dangling conformanceTest rc=0 rc=1

34 checks. make validate green. No other role field in the repo shares this schema, and an estate-wide scan of prophet-mesh / prophet-platform / Noetica found no ABB role outside the enum.

#223 shipped three instances of the defect class it was written to close.

1. The vendor-role control never looked at a document. It built one
   (`e = deepcopy(example); e["role"] = v`) and then never used it, matching
   its regex against six hardcoded strings instead. With `role: "postgres"`
   in the shipped example the validator exited 0 AND printed
   "OK 6 vendor-named roles caught by the deny list". A control that reports
   catching what it has not looked at is worse than an absent one, because it
   produces evidence.

2. `role` was `{type: string, minLength: 1}`. A deny list cannot prevent the
   failure that actually compounds — the same role spelled several ways.
   macOS ships 627 sandbox containers whose extension-point names include
   `extension` (45x), `Extension` (2x), `DiagnosticExtension` (8x),
   `diagnosticextension` (5x) and `diagnostic` (3x): fifteen years of a
   freeform role field with no grammar, after which the name can no longer
   group anything and a side-channel record has to carry the contract.

3. `conformanceTest` pointed at `conformance/abb-database-vectors.json`,
   which does not exist. `format: uri-reference` is a syntax check and
   resolves nothing, so the dangling pointer validated clean.

Changes:

- `role` is a closed enum over the four roles actually in use (prophet-mesh
  specs/abb-catalog.yaml plus the example). Rejection now happens in the
  schema, which every consumer already validates against, not in a private
  regex. Adding a role is a spec PR — the reviewed gate whose absence
  produced the five spellings above.
- The deny list is deleted. Its cases are now fed to the schema AS DOCUMENTS:
  8 vendor names, 6 case/whitespace variants, 1 invented role, all rejected.
- INVARIANT 1a mutation-tests the enum. Without it the cases above would
  keep passing if `role` reverted to a freeform string while some unrelated
  constraint did the rejecting. Drop the enum, confirm `postgres` is
  accepted; restore, confirm rejected.
- INVARIANT 3 resolves `conformanceTest` against the repo, with a negative
  control proving a dangling pointer is detected. The example's dangling
  pointer is removed rather than backfilled with invented vectors: absence is
  documented as meaningful, a fabricated suite would be a new paper control.

Verified by probe, not by exit code: the exact document that passed before
(`role: postgres`) now fails, as do `role: database` and a dangling
conformanceTest. 34 checks, `make validate` green.
Copilot AI review requested due to automatic review settings July 30, 2026 08:25

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR hardens the ArchitecturalBuildingBlock (ABB) contract by moving role validation into the canonical schema (closed enum) and strengthening validator invariants so invalid roles and dangling conformance pointers reliably fail validation.

Changes:

  • Converted role from a freeform non-empty string to a closed enum in schemas/ArchitecturalBuildingBlock.json.
  • Replaced the prior regex-based vendor-role deny check with schema-backed document mutations plus an enum “mutation test” in tools/validate_architectural_building_block.py.
  • Removed the example’s dangling conformanceTest pointer and added validator enforcement that conformanceTest must resolve (or be absent).

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File Description
tools/validate_architectural_building_block.py Removes ineffective regex deny-list logic; adds schema-backed invariants for enum enforcement and conformanceTest resolution.
schemas/ArchitecturalBuildingBlock.json Makes role a closed enum and expands documentation around role/conformanceTest semantics.
examples/architectural-building-block.example.json Drops a dangling conformanceTest pointer from the shipped example.

"type": "string",
"format": "uri-reference",
"description": "Optional pointer to a conformance test suite (like sourceos-spec/conformance/lawful-verdict-vectors.json for LawfulDispatchReceipt). Two implementations of the same ABB that each pass their own unit tests can still disagree with each other on the ABB's contract — only shared vectors detect that. Present ⇒ implementations are held to it; absent ⇒ conformance is per-implementation and drift is discoverable only by breaking behaviour."
"description": "Optional pointer to a conformance test suite (like sourceos-spec/conformance/lawful-verdict-vectors.json for LawfulDispatchReceipt). Two implementations of the same ABB that each pass their own unit tests can still disagree with each other on the ABB's contract — only shared vectors detect that. Present ⇒ implementations are held to it; absent ⇒ conformance is per-implementation and drift is discoverable only by breaking behaviour.\n\n`format: uri-reference` is a SYNTAX check and resolves nothing — this field shipped pointing at `conformance/abb-database-vectors.json`, a file that does not exist, and validated clean. Repo-relative pointers are therefore resolved by tools/validate_pointers_resolve.py, which fails on a dangling one. A pointer to a suite that is not there is worse than absence: absence is documented above as meaningful, whereas a dangling pointer reads as 'held to shared vectors' to every consumer that does not go looking."
Comment on lines +161 to +171
ptr = example.get("conformanceTest")
if ptr is None:
print(" OK example declares no conformanceTest — absence is meaningful per schema "
"(conformance is per-implementation until shared vectors exist)")
elif (REPO_ROOT / ptr).is_file():
print(f" OK {ptr} resolves")
else:
failures.append(
f"conformanceTest {ptr!r} does not resolve to a file under {REPO_ROOT}. A pointer to "
"a suite that is not there is worse than absence — drop the field or add the suite."
)
@mdheller
mdheller merged commit ff81a3d into main Jul 30, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants