Skip to content

Commit 7fbae09

Browse files
bokelleyclaude
andcommitted
feat(webhooks): HMAC/bearer/Docker/Standard Webhooks delivery modes
Closes #478. Three additive features on the existing WebhookSender stack: - WebhookSender.from_bearer_token / from_adcp_legacy_hmac / from_standard_webhooks_secret — alt-auth modes for buyers who authenticate at the gateway, run AdCP-3.x HMAC, or run Svix/Resend. from_adcp_legacy_hmac emits a one-shot DeprecationWarning mirroring the receiver-side warn-once so sender-only operators see the AdCP 4.0 cutover signal at runtime. - TransportHook Protocol + DockerLocalhostRewrite — opt-in URL rewrite before SSRF for senders running inside Docker. Hooks run pre-SSRF; SSRF stays authoritative on the rewritten URL. DockerLocalhostRewrite raises at sender construction unless allow_private_destinations=True so the rewrite isn't a silent no-op. apply_hooks rejects scheme/port changes so a hook cannot widen authority beyond hostname rewrite. - adcp.signing.standard_webhooks — pure-Python standardwebhooks.com v1 primitives with svix-Python interop tested both directions. Public SECRET_PREFIX constant lets test fixtures construct the canonical whsec_<base64> form via import (rather than embedding the literal pattern, which trips high-entropy-secret detectors on test data). decode_secret error path elides the underlying binascii exception message — that string can echo a fragment of the offending secret into operator logs. Internal: extracted WebhookAuthStrategy Protocol + 4 strategies (JWK / AdCP-legacy HMAC / Standard Webhooks / bearer). _send_bytes calls self._auth.build_auth_headers per request — HMAC modes naturally re-sign on resend() with a fresh timestamp, satisfying the receiver- side 300s skew window. Strategies use field(repr=False) on secret- bearing fields so the auto-generated dataclass repr cannot leak credentials via tracebacks / vars() / faulthandler. 51 new tests (49 in-process + 2 env-gated Svix sandbox). Migration doc at docs/webhooks/migration-from-fragmented-senders.md translates common legacy sender patterns to the consolidated API. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 9985086 commit 7fbae09

12 files changed

Lines changed: 2089 additions & 55 deletions
Lines changed: 274 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,274 @@
1+
# Migrating from fragmented webhook senders
2+
3+
If your operator stack has multiple legacy webhook-sending paths — `requests.post`-and-handroll-the-headers in one service, a homemade HMAC helper in another, ad-hoc retries in a third — `WebhookSender` consolidates them. Same one-call delivery API across every authentication mode supported by AdCP and by common buyer ecosystems (Svix/Resend/operator-bearer).
4+
5+
This guide is a translation table. Find your legacy pattern on the left; the right column is the equivalent on `WebhookSender`.
6+
7+
The killer property: in every row, the bytes the sender signs are byte-for-byte the bytes that hit the wire. The classic "I signed `json.dumps(payload, separators=(',',':'))` but `requests` re-serialized with whitespace before POST" bug is impossible by construction.
8+
9+
---
10+
11+
## Auth modes
12+
13+
`WebhookSender` exposes one classmethod per auth mode. Pick one per sender; if you need two modes (bearer-at-gateway plus end-to-end signature), construct two senders.
14+
15+
### RFC 9421 JWK signing — the AdCP-conformant default
16+
17+
```python
18+
from adcp.webhooks import WebhookSender
19+
20+
sender = WebhookSender.from_jwk(webhook_signing_jwk_with_private_d)
21+
async with sender:
22+
result = await sender.send_mcp(
23+
url="https://buyer.example.com/webhooks/adcp/create_media_buy/op_abc",
24+
task_id="task_456",
25+
task_type="create_media_buy",
26+
status="completed",
27+
result={"media_buy_id": "mb_1"},
28+
)
29+
```
30+
31+
Use this for every AdCP-conformant buyer. JWK signing is the spec baseline.
32+
33+
### `Authorization: Bearer <token>` — for buyers who authenticate at the gateway
34+
35+
```python
36+
sender = WebhookSender.from_bearer_token("super-secret-token")
37+
async with sender:
38+
result = await sender.send_mcp(url=..., task_id=..., status=...)
39+
```
40+
41+
The body still goes through the same byte-exact marshaling, and `idempotency_key` still ends up inside the JSON for receiver dedup. There is no body signature; a buyer treating the bearer as the sole authenticity signal must enforce TLS pinning or mTLS at the transport layer to make a stolen token unusable.
42+
43+
### AdCP-legacy HMAC-SHA256 — back-compat for AdCP 3.x receivers
44+
45+
```python
46+
sender = WebhookSender.from_adcp_legacy_hmac(
47+
secret=b"shared-secret-bytes",
48+
key_id="kid_buyer_42",
49+
)
50+
async with sender:
51+
result = await sender.send_mcp(url=..., task_id=..., status=...)
52+
```
53+
54+
Wire format matches `verify_webhook_hmac` in `adcp.signing.webhook_hmac`: `X-AdCP-Signature: sha256=<hex>` over `f"{timestamp}.{body}"`, with `X-AdCP-Timestamp` set fresh on every delivery (resends produce a new signature over the same body bytes — receivers enforcing a 300s skew window won't reject the retry).
55+
56+
Plan to migrate to JWK signing before AdCP 4.0; the legacy `authentication` field is removed in 4.0.
57+
58+
### Standard Webhooks v1 — Svix / Resend / standardwebhooks.com interop
59+
60+
```python
61+
sender = WebhookSender.from_standard_webhooks_secret(
62+
"whsec_<base64-distributed-by-buyer>",
63+
key_id="kid_svix_42",
64+
)
65+
async with sender:
66+
result = await sender.send_mcp(url=..., task_id=..., status=...)
67+
```
68+
69+
The constructor takes the canonical `whsec_<base64>` form Svix and Resend distribute and base64-decodes it internally. **Do not pass the literal `whsec_…` string to `from_adcp_legacy_hmac`** — the AdCP-legacy scheme HMACs against raw bytes, so you'd silently produce signatures Svix would reject. The two constructors enforce different secret-encoding contracts at the type level so this swap can't happen.
70+
71+
Wire format per spec: `webhook-id` / `webhook-timestamp` / `webhook-signature: v1,<base64>` over `f"{webhook_id}.{webhook_timestamp}.{body}"`. Each delivery gets a fresh `webhook-id` so a Svix-style receiver caching ids for replay defense doesn't false-positive on a legitimate retry.
72+
73+
---
74+
75+
## Translation table
76+
77+
### Pattern: hand-built bearer POST
78+
79+
```python
80+
# Legacy
81+
import requests, json
82+
requests.post(
83+
url,
84+
data=json.dumps(payload),
85+
headers={
86+
"Authorization": f"Bearer {token}",
87+
"Content-Type": "application/json",
88+
},
89+
)
90+
```
91+
92+
```python
93+
# Equivalent
94+
sender = WebhookSender.from_bearer_token(token)
95+
async with sender:
96+
await sender.send_raw(
97+
url=url,
98+
idempotency_key=payload["idempotency_key"],
99+
payload=payload,
100+
)
101+
```
102+
103+
`send_raw` requires `idempotency_key` as a kwarg — it's injected into the payload before serialization. Pass the same value you used to use; if your legacy code generated it ad hoc, call `generate_webhook_idempotency_key()` once and pass it in.
104+
105+
### Pattern: hand-built HMAC POST with the sign-vs-body bug
106+
107+
```python
108+
# Legacy — note the lurking serialization mismatch
109+
import hmac, hashlib, json, requests
110+
body_str = json.dumps(payload, separators=(",", ":"))
111+
sig = hmac.new(secret, body_str.encode(), hashlib.sha256).hexdigest()
112+
requests.post(
113+
url,
114+
json=payload, # ← reserializes with default separators; signature mismatches
115+
headers={"X-AdCP-Signature": f"sha256={sig}", ...},
116+
)
117+
```
118+
119+
```python
120+
# Equivalent — sender signs and sends the same bytes
121+
sender = WebhookSender.from_adcp_legacy_hmac(secret, key_id="kid_buyer_42")
122+
async with sender:
123+
await sender.send_raw(
124+
url=url,
125+
idempotency_key=payload["idempotency_key"],
126+
payload=payload,
127+
)
128+
```
129+
130+
The "sign one byte sequence, POST a different one via `json=`" bug is one of the most common HMAC-webhook integration failures in the field. `WebhookSender` serializes once with `json.dumps(...)` and POSTs the same bytes via `httpx.AsyncClient.post(content=body)` — the bug class is impossible.
131+
132+
### Pattern: hand-built Standard-Webhooks (Svix-style) POST
133+
134+
```python
135+
# Legacy
136+
import base64, hmac, hashlib, json, time, requests, uuid
137+
secret_bytes = base64.b64decode(secret_str.removeprefix("whsec_") + "==")
138+
msg_id = f"msg_{uuid.uuid4().hex}"
139+
ts = str(int(time.time()))
140+
body = json.dumps(payload, separators=(",", ":")).encode()
141+
mac = hmac.new(secret_bytes, f"{msg_id}.{ts}.".encode() + body, hashlib.sha256).digest()
142+
requests.post(
143+
url,
144+
data=body,
145+
headers={
146+
"webhook-id": msg_id,
147+
"webhook-timestamp": ts,
148+
"webhook-signature": f"v1,{base64.b64encode(mac).decode()}",
149+
"Content-Type": "application/json",
150+
},
151+
)
152+
```
153+
154+
```python
155+
# Equivalent
156+
sender = WebhookSender.from_standard_webhooks_secret(secret_str, key_id="kid")
157+
async with sender:
158+
await sender.send_raw(
159+
url=url,
160+
idempotency_key=payload["idempotency_key"],
161+
payload=payload,
162+
)
163+
```
164+
165+
### Pattern: custom Docker localhost rewrite
166+
167+
```python
168+
# Legacy — homemade rewrite for "deliver from Docker container to host webhook"
169+
url = url.replace("localhost", "host.docker.internal")
170+
url = url.replace("127.0.0.1", "host.docker.internal")
171+
```
172+
173+
```python
174+
# Equivalent — TransportHook composes with everything else
175+
from adcp.webhook_sender import WebhookSender
176+
from adcp.webhook_transport_hooks import DockerLocalhostRewrite
177+
178+
sender = WebhookSender.from_jwk(
179+
jwk,
180+
transport_hooks=(DockerLocalhostRewrite(),),
181+
allow_private_destinations=True, # required — see note below
182+
)
183+
```
184+
185+
`DockerLocalhostRewrite` requires `allow_private_destinations=True` at sender construction — the rewrite produces a private-IP destination, and SSRF would reject the rewritten URL otherwise. The flag is the operator's explicit opt-in; it raises at construction time if forgotten so the misconfiguration doesn't bite at first delivery.
186+
187+
For Linux containers without `--add-host=host.docker.internal:host-gateway`, pass `DockerLocalhostRewrite(rewrite_to="172.17.0.1")` (Docker's default bridge gateway).
188+
189+
### Pattern: per-call retry loop with backoff
190+
191+
```python
192+
# Legacy
193+
import time
194+
for attempt in range(5):
195+
try:
196+
response = requests.post(url, ...)
197+
if response.status_code < 500:
198+
break
199+
except requests.RequestException:
200+
pass
201+
time.sleep(2 ** attempt)
202+
```
203+
204+
```python
205+
# Equivalent — WebhookDeliverySupervisor owns retry + circuit breaker + dedup
206+
from adcp.webhooks import (
207+
InMemoryWebhookDeliverySupervisor,
208+
WebhookDeliveryRequest,
209+
)
210+
211+
supervisor = InMemoryWebhookDeliverySupervisor(
212+
sender=sender,
213+
# tune retry policy / circuit-breaker thresholds at construction
214+
)
215+
await supervisor.deliver(WebhookDeliveryRequest(url=..., payload=..., ...))
216+
```
217+
218+
The supervisor handles retry policy, circuit-breaker state per buyer, and durable queueing. For Postgres-backed deployments, swap in `PgWebhookDeliverySupervisor` with the same Protocol — the calling code doesn't change.
219+
220+
### Pattern: per-call SSRF check
221+
222+
```python
223+
# Legacy
224+
host = urlparse(url).hostname
225+
ip = socket.gethostbyname(host)
226+
if ipaddress.ip_address(ip).is_private:
227+
raise ValueError("private IP not allowed")
228+
requests.post(url, ...) # ← TOCTOU: DNS may rebind between check and connect
229+
```
230+
231+
```python
232+
# Equivalent — automatic. WebhookSender pins the validated IP into the
233+
# transport so DNS rebinding cannot swap the connect target between
234+
# validation and POST.
235+
sender = WebhookSender.from_jwk(jwk) # SSRF runs on every send
236+
```
237+
238+
The owned-client path rebuilds an `AsyncIpPinnedTransport` per request, runs the full SSRF range check (loopback / RFC 1918 / link-local / CGNAT / IPv6 ULA / multicast / cloud metadata), enforces an optional port allowlist, and pins the connection to the validated IP. There is no "remembered to call the SSRF helper" failure mode.
239+
240+
If your infrastructure uses a vetted egress proxy with mTLS to a fixed buyer set, pass your own `client=httpx.AsyncClient(...)` and the sender will trust the operator's transport instead.
241+
242+
### Pattern: ad-hoc retry that re-signs with the original timestamp
243+
244+
```python
245+
# Legacy — replays the original signature, which receivers reject after
246+
# the 300s skew window.
247+
saved = (signed_headers, body_bytes)
248+
time.sleep(60)
249+
requests.post(url, data=saved[1], headers=saved[0])
250+
```
251+
252+
```python
253+
# Equivalent — WebhookSender.resend re-signs the same bytes with a
254+
# fresh signature on every retry, preserving idempotency_key for dedup.
255+
result = await sender.send_mcp(url=..., task_id=..., status=...)
256+
if not result.ok:
257+
retry = await sender.resend(result)
258+
```
259+
260+
`resend` works in every auth mode. JWK regenerates the 9421 Signature/Signature-Input headers; HMAC modes generate a new timestamp + signature over the same body; bearer mode is a no-op (the auth header is timestamp-independent).
261+
262+
---
263+
264+
## Failure modes the new API closes
265+
266+
* **Sign-vs-body divergence** — the sender owns marshaling; signed bytes equal sent bytes.
267+
* **Retry stales out the signature**`resend` re-signs.
268+
* **Forgotten `idempotency_key`** — required kwarg on `send_raw`; receivers dedupe on it.
269+
* **DNS rebinding between SSRF check and POST** — automatic IP-pin on the owned-client path.
270+
* **Standard Webhooks secret encoding mistake** — typed split between `from_adcp_legacy_hmac(bytes)` and `from_standard_webhooks_secret(str)`.
271+
* **Docker rewrite without operator opt-in**`DockerLocalhostRewrite` raises at construction unless `allow_private_destinations=True`.
272+
* **Hooks bypassing SSRF** — hooks run before SSRF, but SSRF validates the post-rewrite URL; the boundary holds.
273+
274+
If your current sender doesn't have one of these properties, this is what you're trading up to.

src/adcp/signing/__init__.py

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -239,6 +239,14 @@
239239
async_sign_request,
240240
sign_request,
241241
)
242+
from adcp.signing.standard_webhooks import (
243+
StandardWebhookError,
244+
sign_standard_webhook,
245+
verify_standard_webhook,
246+
)
247+
from adcp.signing.standard_webhooks import (
248+
decode_secret as decode_standard_webhook_secret,
249+
)
242250
from adcp.signing.verifier import (
243251
VerifiedSigner,
244252
VerifierCapability,
@@ -343,6 +351,7 @@ def __init__(self, *args: object, **kwargs: object) -> None:
343351
"SignatureInputLabel",
344352
"SignatureVerificationError",
345353
"SignedHeaders",
354+
"StandardWebhookError",
346355
"SigningAlgorithm",
347356
"SigningConfig",
348357
"SigningDecision",
@@ -371,6 +380,7 @@ def __init__(self, *args: object, **kwargs: object) -> None:
371380
"canonicalize_target_uri",
372381
"compute_content_digest_sha256",
373382
"content_digest_matches",
383+
"decode_standard_webhook_secret",
374384
"default_capability_cache",
375385
"default_jwks_fetcher",
376386
"default_revocation_list_fetcher",
@@ -389,6 +399,7 @@ def __init__(self, *args: object, **kwargs: object) -> None:
389399
"resolve_and_validate_host",
390400
"sign_request",
391401
"sign_signature_base",
402+
"sign_standard_webhook",
392403
"signing_operation",
393404
"unauthorized_response_headers",
394405
"validate_jwks_uri",
@@ -398,5 +409,6 @@ def __init__(self, *args: object, **kwargs: object) -> None:
398409
"verify_jws_document",
399410
"verify_request_signature",
400411
"verify_signature",
412+
"verify_standard_webhook",
401413
"verify_starlette_request",
402414
]

0 commit comments

Comments
 (0)