docs(compliance): single doc explaining how compliance grading works end-to-end

[bokelley]

Motivation

The compliance grading model is sophisticated but undocumented as a system. An adopter (or SDK contributor) trying to understand "what scenarios will my declared specialism actually be graded on?" has to grep across:

• `static/compliance/source/specialisms//index.yaml` — the specialism manifest with `requires_scenarios`
• `static/compliance/source/protocols//scenarios/.yaml` — the per-scenario fixture
• Capability flags (e.g., `media_buy.supports_proposals`) that gate scenario applicability via `comply_scenario` references
• The CLI invocation `adcp storyboard run media_buy_seller` which loads from the above

I (a Python SDK contributor implementing the seller-side framework) spent over an hour misreading the system because I was looking at `src/lib/testing/scenarios/media-buy.ts` (JS-internal test code that always uses `buying_mode='brief'`) and missed that the authoritative storyboards are the YAML compliance specs that DO drive `buying_mode='refine'` + `action='finalize'`. This led me to incorrectly claim the proposal lifecycle wasn't being exercised when in fact it is.

What's missing

A single doc page (suggested location: `docs/compliance/grading-model.md` or `docs/compliance/index.mdx`) that walks through:

1. Specialism declaration — how an adopter declares specialisms in `DecisioningCapabilities.specialisms` and what that commits them to.
2. Scenario resolution — how the storyboard runner takes `media_buy_seller` + the agent's declared specialisms and produces the resolved `requires_scenarios` list.
3. Capability gates — how flags like `media_buy.supports_proposals` skip scenarios as `capability_unsupported` vs grade them.
4. Result interpretation — how to read `overall_status` (`passed` / `partial` / `failed`), what `tracks_passed` / `steps_passed` mean, and how to find which exact scenario failed.
5. Adding a new scenario — for spec contributors: where the YAML lives, what fields the runner consumes, naming conventions.

Why this matters

• New SDK contributors hit this discoverability wall every time. The cost is each one spending hours triangulating the system.
• Adopters porting from monolithic codebases (e.g., the Python SDK is currently mid-port from a single example to multi-platform) need to know what compliance gates they're committing to when they declare a specialism.
• A clear doc would have prevented at least one in-flight architectural mischaracterization in adcp-client-python.

Acceptance criteria

• Single doc page covering the five items above
• Examples showing the full grading flow for at least one specialism (sales-guaranteed is a rich one — `requires_scenarios` includes `proposal_finalize`, `refine_products`, `pending_creatives_to_start`)
• Cross-links from each specialism's `index.yaml` and key scenarios back to this doc
• Cross-links from the relevant CLI commands (`adcp storyboard run`, `adcp storyboard show`)

Refs

• Currently scattered: `static/compliance/source/specialisms/sales-guaranteed/index.yaml`, `static/compliance/source/protocols/media-buy/scenarios/proposal_finalize.yaml`, capability-flag definitions in `static/schemas/source/...`
• This issue surfaced while implementing ProposalManager v1 in adcp-client-python (PR Fix duplicate const declaration in dashboard.html #504); discoverability gap was load-bearing for that work

[bokelley]

Triage

Classification: Doc addition
Bucket(s): web / site / docs, compliance-suite
Status: drafting-pr

What the experts said:

• docs-expert: placement is docs/building/verification/grading-model.mdx (not a new docs/compliance/ section); audience split works with items 1–4 for adopter-operators and contributor path handled via Related link to storyboard-authoring.md; two-phase scenario merge (protocol baseline + specialism requires_scenarios) and capability_unsupported gate output shape must be shown
• ad-tech-protocol-expert: overall_status values are passing/failing/partial (not passed/failed/partial); DecisioningCapabilities is not a real type — declaration is via get_adcp_capabilities response specialisms field; invariants: [status.monotonic] is a separate failure axis that must be covered; change is non-breaking, zero wire impact

My take: All five acceptance criteria covered, both expert blockers fixed before PR open. Draft PR #4040 is open — adds docs/building/verification/grading-model.mdx with accurate field names sourced from compliance-db.ts and the specialism/scenario YAMLs.

Drafting PR: #4040 — docs(compliance): add compliance grading model reference page

---

Triaged by Claude Code. Session: https://claude.ai/code/session_01CPn4oJdWTTVZ2L8WCD4pDg

---

Generated by Claude Code