ABB: role becomes a closed enum, and the vendor control gets teeth - #225
Merged
Conversation
#223 shipped three instances of the defect class it was written to close. 1. The vendor-role control never looked at a document. It built one (`e = deepcopy(example); e["role"] = v`) and then never used it, matching its regex against six hardcoded strings instead. With `role: "postgres"` in the shipped example the validator exited 0 AND printed "OK 6 vendor-named roles caught by the deny list". A control that reports catching what it has not looked at is worse than an absent one, because it produces evidence. 2. `role` was `{type: string, minLength: 1}`. A deny list cannot prevent the failure that actually compounds — the same role spelled several ways. macOS ships 627 sandbox containers whose extension-point names include `extension` (45x), `Extension` (2x), `DiagnosticExtension` (8x), `diagnosticextension` (5x) and `diagnostic` (3x): fifteen years of a freeform role field with no grammar, after which the name can no longer group anything and a side-channel record has to carry the contract. 3. `conformanceTest` pointed at `conformance/abb-database-vectors.json`, which does not exist. `format: uri-reference` is a syntax check and resolves nothing, so the dangling pointer validated clean. Changes: - `role` is a closed enum over the four roles actually in use (prophet-mesh specs/abb-catalog.yaml plus the example). Rejection now happens in the schema, which every consumer already validates against, not in a private regex. Adding a role is a spec PR — the reviewed gate whose absence produced the five spellings above. - The deny list is deleted. Its cases are now fed to the schema AS DOCUMENTS: 8 vendor names, 6 case/whitespace variants, 1 invented role, all rejected. - INVARIANT 1a mutation-tests the enum. Without it the cases above would keep passing if `role` reverted to a freeform string while some unrelated constraint did the rejecting. Drop the enum, confirm `postgres` is accepted; restore, confirm rejected. - INVARIANT 3 resolves `conformanceTest` against the repo, with a negative control proving a dangling pointer is detected. The example's dangling pointer is removed rather than backfilled with invented vectors: absence is documented as meaningful, a fabricated suite would be a new paper control. Verified by probe, not by exit code: the exact document that passed before (`role: postgres`) now fails, as do `role: database` and a dangling conformanceTest. 34 checks, `make validate` green.
There was a problem hiding this comment.
Pull request overview
This PR hardens the ArchitecturalBuildingBlock (ABB) contract by moving role validation into the canonical schema (closed enum) and strengthening validator invariants so invalid roles and dangling conformance pointers reliably fail validation.
Changes:
- Converted
rolefrom a freeform non-empty string to a closedenuminschemas/ArchitecturalBuildingBlock.json. - Replaced the prior regex-based vendor-role deny check with schema-backed document mutations plus an enum “mutation test” in
tools/validate_architectural_building_block.py. - Removed the example’s dangling
conformanceTestpointer and added validator enforcement thatconformanceTestmust resolve (or be absent).
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| tools/validate_architectural_building_block.py | Removes ineffective regex deny-list logic; adds schema-backed invariants for enum enforcement and conformanceTest resolution. |
| schemas/ArchitecturalBuildingBlock.json | Makes role a closed enum and expands documentation around role/conformanceTest semantics. |
| examples/architectural-building-block.example.json | Drops a dangling conformanceTest pointer from the shipped example. |
| "type": "string", | ||
| "format": "uri-reference", | ||
| "description": "Optional pointer to a conformance test suite (like sourceos-spec/conformance/lawful-verdict-vectors.json for LawfulDispatchReceipt). Two implementations of the same ABB that each pass their own unit tests can still disagree with each other on the ABB's contract — only shared vectors detect that. Present ⇒ implementations are held to it; absent ⇒ conformance is per-implementation and drift is discoverable only by breaking behaviour." | ||
| "description": "Optional pointer to a conformance test suite (like sourceos-spec/conformance/lawful-verdict-vectors.json for LawfulDispatchReceipt). Two implementations of the same ABB that each pass their own unit tests can still disagree with each other on the ABB's contract — only shared vectors detect that. Present ⇒ implementations are held to it; absent ⇒ conformance is per-implementation and drift is discoverable only by breaking behaviour.\n\n`format: uri-reference` is a SYNTAX check and resolves nothing — this field shipped pointing at `conformance/abb-database-vectors.json`, a file that does not exist, and validated clean. Repo-relative pointers are therefore resolved by tools/validate_pointers_resolve.py, which fails on a dangling one. A pointer to a suite that is not there is worse than absence: absence is documented above as meaningful, whereas a dangling pointer reads as 'held to shared vectors' to every consumer that does not go looking." |
Comment on lines
+161
to
+171
| ptr = example.get("conformanceTest") | ||
| if ptr is None: | ||
| print(" OK example declares no conformanceTest — absence is meaningful per schema " | ||
| "(conformance is per-implementation until shared vectors exist)") | ||
| elif (REPO_ROOT / ptr).is_file(): | ||
| print(f" OK {ptr} resolves") | ||
| else: | ||
| failures.append( | ||
| f"conformanceTest {ptr!r} does not resolve to a file under {REPO_ROOT}. A pointer to " | ||
| "a suite that is not there is worse than absence — drop the field or add the suite." | ||
| ) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
#223 shipped three instances of the defect class it was written to close. Found by turning today's Apple-container lens back on our own spec.
1. The vendor-role control never looked at a document
It built one and then never used it:
So the regex was only ever tested against its own six fixtures. With
role: "postgres"in the shipped example, the validator exited 0 and printedOK 6 vendor-named roles caught by the deny list. A control that reports catching what it has not looked at is worse than an absent one — it produces evidence.2. A deny list cannot prevent the failure that compounds
rolewas{type: string, minLength: 1}. Vendor names are the obvious slip; the one that actually accumulates is the same role spelled several ways. Measured on macOS 26.3.1 — 627 sandbox containers, extension-point names including:extensionDiagnosticExtensiondiagnosticextensiondiagnosticExtensionFifteen years of a freeform role field with no enforced grammar. Past that point the name cannot group anything and a side-channel record has to carry the contract instead. An allowlist forecloses both failures at once, and forecloses them in the schema — the artefact every consumer already validates against, unlike a validator's private regex.
3.
conformanceTestpointed at a file that does not existconformance/abb-database-vectors.jsonis not in the repo.format: uri-referenceis a syntax check and resolves nothing, so the dangling pointer validated clean — and read as "held to shared vectors" to every consumer that did not go looking.What changed
roleis a closed enum over the four roles actually in use (prophet-meshspecs/abb-catalog.yaml+ the example), never roles we imagine wanting. Adding one is a spec PR — the reviewed gate whose absence produced the five spellings above.rolereverts to a freeform string while some unrelated constraint does the rejecting. Drop the enum ⇒postgresaccepted; restore ⇒ rejected.Verified by probe, not by exit code
role: database(case variant)conformanceTest34 checks.
make validategreen. No otherrolefield in the repo shares this schema, and an estate-wide scan of prophet-mesh / prophet-platform / Noetica found no ABB role outside the enum.