docs(compliance): explain how grading works end-to-end (#4037)#4044
Draft
docs(compliance): explain how grading works end-to-end (#4037)#4044
Conversation
Add docs/building/verification/how-grading-works.mdx explaining how the runner translates a specialism declaration into a concrete set of graded scenarios and evaluates per-scenario requires_capability gates. Also corrects the sales-guaranteed specialism narrative which incorrectly stated that omitting supports_proposals would trigger proposal grading; the schema default is false so absent = skip (not grade). https://claude.ai/code/session_01M71brR4fRwi8w1CdjZUXXR
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #4037
Adds
docs/building/verification/how-grading-works.mdx— a new page explaining how the compliance runner translates a specialism declaration into a concrete set of graded scenarios, and how per-scenariorequires_capabilitygates narrow that set. Also corrects a factual error in thesales-guaranteedspecialism narrative and adds a cross-link from the Compliance Catalog.Non-breaking justification: New docs page (additive), one-sentence cross-link addition to compliance-catalog.mdx, narrative-prose correction in a specialism YAML (no schema or wire changes), nav entry in docs.json. Existing consumers unaffected.
Factual correction: The
sales-guaranteedspecialism narrative previously stated "sellers that declaretrue(or omit the field) are graded againstproposal_finalize." The capabilities schema has"default": falseforsupports_proposalswith description "When false or absent, conformance runners skip proposal-lifecycle storyboards." The narrative conflicted with the schema. The corrected narrative reflects the schema: omit = false = skip. A previous triage commit on this branch documented absence astrue; this commit supersedes it with the schema-accurate behavior.Key content:
requires_scenarioslist → per-scenariorequires_capabilitygate → skip or gradesales-guaranteed+media_buy.supports_proposals: true(all 8 scenarios run) vsfalse(7 run,proposal_finalizeskipped)skip.reason: not_applicableis the canonical runner code for skipped scenarios —capability_unsupportedonly appears in narrative prose and is not a runner output contract valuerunner-output-contract.yamlNits not fixed (surfaced for reviewer):
requires_capabilityfield (it's used in one scenario,proposal_finalize, but not in storyboard-schema.yaml). The page notes this is a nascent mechanism./schemas/v3/protocol/get-adcp-capabilities-response.jsonuses the major-version alias per playbook convention;$idin the schema file uses a versionless path — both are correct for their respective purposes.Pre-PR review:
passed: null(not in contract spec) → removed in favor ofskipblock as authoritative signal; fixednot_applicabletable to consolidate cases withskip.detailas the distinguisher<Note>component correctSession: https://claude.ai/code/session_01M71brR4fRwi8w1CdjZUXXR
Generated by Claude Code