Problem
Per AdCP, every seller has to expose either list_accounts or sync_accounts — it's how buyers know who they can transact as. An agent that exposes neither has a real compliance gap on the account-management surface, not a "we'll grade you later when you implement it" deferral.
The storyboard runner currently treats missing-account-tool as a skip, with the implicit "exemption" message:
warnings: [
"Agent did not advertise tool \"sync_accounts\"; agent tools: [...]. Sole stateful
step exemption applied for phase 'setup': no peer could have established substitute
state, so downstream phases run without cascade (adcp#4053, adcp-client#1146/#1545)."
]
skip_reason: "missing_tool"
Translation: "I would have failed you, but I let it slide because nothing else could have set the state up."
That exemption logic makes sense for optional capability scenarios (e.g. axe_signals — only graded if the seller advertises it). It does not make sense for tools the spec mandates as a one-of-N choice. A real launch-readiness check should call those out.
Repro
Run media_buy_seller against an agent that exposes neither sync_accounts nor list_accounts:
── A2A ──
Steps: 31 passed, 0 failed, 28 skipped — STORYBOARD-OK
The agent passes compliance despite missing a required-by-spec capability. That's the wrong signal.
Proposal
For tools that AdCP marks as required (or required-as-one-of-N), the storyboard runner should:
- Inspect agent capabilities once at run start.
- For each spec-required tool family, check that at least one member is advertised.
- Emit a hard failure (
STORYBOARD-FAIL line, exit 1, contributing to the failed count) when a required family is empty — distinct from per-scenario skips.
Suggested error shape in the summary:
── A2A ──
Steps: 31 passed, 1 failed, 28 skipped
Failing checks:
- capability:account_management — agent advertises neither sync_accounts nor list_accounts
Spec: every seller must expose at least one account-discovery tool
Affected: refine_products/setup (skipped due to this gap)
Where to encode the requirement
Two options, increasing rigour:
- Hard-coded in the runner for the small set of required-one-of-N families known today (
account_management, possibly creative_management).
- Driven by the manifest/spec —
error_code_policy already has structured metadata; an analogous tool_requirements block on get_adcp_capabilities could declare which tool sets are required-one-of-N. The runner reads it and grades accordingly.
Option 2 is the better long-term shape but option 1 unblocks today's compliance signal.
Why it matters
Without this, an agent can claim AdCP compliance while missing capabilities the spec mandates — the buyer side has no protocol-level way to discover the gap. The storyboard is supposed to be the launch-readiness gate; it should grade the spec, not the agent's self-declaration.
Related: #1623 (skip-cause aggregation in the summary) — this issue is about reclassifying which skips become fails; that one is about surfacing skip causes regardless of classification.
Problem
Per AdCP, every seller has to expose either
list_accountsorsync_accounts— it's how buyers know who they can transact as. An agent that exposes neither has a real compliance gap on the account-management surface, not a "we'll grade you later when you implement it" deferral.The storyboard runner currently treats missing-account-tool as a skip, with the implicit "exemption" message:
Translation: "I would have failed you, but I let it slide because nothing else could have set the state up."
That exemption logic makes sense for optional capability scenarios (e.g.
axe_signals— only graded if the seller advertises it). It does not make sense for tools the spec mandates as a one-of-N choice. A real launch-readiness check should call those out.Repro
Run
media_buy_selleragainst an agent that exposes neithersync_accountsnorlist_accounts:The agent passes compliance despite missing a required-by-spec capability. That's the wrong signal.
Proposal
For tools that AdCP marks as required (or required-as-one-of-N), the storyboard runner should:
STORYBOARD-FAILline, exit 1, contributing to the failed count) when a required family is empty — distinct from per-scenario skips.Suggested error shape in the summary:
Where to encode the requirement
Two options, increasing rigour:
account_management, possiblycreative_management).error_code_policyalready has structured metadata; an analogoustool_requirementsblock onget_adcp_capabilitiescould declare which tool sets are required-one-of-N. The runner reads it and grades accordingly.Option 2 is the better long-term shape but option 1 unblocks today's compliance signal.
Why it matters
Without this, an agent can claim AdCP compliance while missing capabilities the spec mandates — the buyer side has no protocol-level way to discover the gap. The storyboard is supposed to be the launch-readiness gate; it should grade the spec, not the agent's self-declaration.
Related: #1623 (skip-cause aggregation in the summary) — this issue is about reclassifying which skips become fails; that one is about surfacing skip causes regardless of classification.