|
| 1 | +# Migrating from fragmented webhook senders |
| 2 | + |
| 3 | +If your operator stack has multiple legacy webhook-sending paths — `requests.post`-and-handroll-the-headers in one service, a homemade HMAC helper in another, ad-hoc retries in a third — `WebhookSender` consolidates them. Same one-call delivery API across every authentication mode supported by AdCP and by common buyer ecosystems (Svix/Resend/operator-bearer). |
| 4 | + |
| 5 | +This guide is a translation table. Find your legacy pattern on the left; the right column is the equivalent on `WebhookSender`. |
| 6 | + |
| 7 | +The killer property: in every row, the bytes the sender signs are byte-for-byte the bytes that hit the wire. The classic "I signed `json.dumps(payload, separators=(',',':'))` but `requests` re-serialized with whitespace before POST" bug is impossible by construction. |
| 8 | + |
| 9 | +--- |
| 10 | + |
| 11 | +## Auth modes |
| 12 | + |
| 13 | +`WebhookSender` exposes one classmethod per auth mode. Pick one per sender; if you need two modes (bearer-at-gateway plus end-to-end signature), construct two senders. |
| 14 | + |
| 15 | +### RFC 9421 JWK signing — the AdCP-conformant default |
| 16 | + |
| 17 | +```python |
| 18 | +from adcp.webhooks import WebhookSender |
| 19 | + |
| 20 | +sender = WebhookSender.from_jwk(webhook_signing_jwk_with_private_d) |
| 21 | +async with sender: |
| 22 | + result = await sender.send_mcp( |
| 23 | + url="https://buyer.example.com/webhooks/adcp/create_media_buy/op_abc", |
| 24 | + task_id="task_456", |
| 25 | + task_type="create_media_buy", |
| 26 | + status="completed", |
| 27 | + result={"media_buy_id": "mb_1"}, |
| 28 | + ) |
| 29 | +``` |
| 30 | + |
| 31 | +Use this for every AdCP-conformant buyer. JWK signing is the spec baseline. |
| 32 | + |
| 33 | +### `Authorization: Bearer <token>` — for buyers who authenticate at the gateway |
| 34 | + |
| 35 | +```python |
| 36 | +sender = WebhookSender.from_bearer_token("super-secret-token") |
| 37 | +async with sender: |
| 38 | + result = await sender.send_mcp(url=..., task_id=..., status=...) |
| 39 | +``` |
| 40 | + |
| 41 | +The body still goes through the same byte-exact marshaling, and `idempotency_key` still ends up inside the JSON for receiver dedup. There is no body signature; a buyer treating the bearer as the sole authenticity signal must enforce TLS pinning or mTLS at the transport layer to make a stolen token unusable. |
| 42 | + |
| 43 | +### AdCP-legacy HMAC-SHA256 — back-compat for AdCP 3.x receivers |
| 44 | + |
| 45 | +```python |
| 46 | +sender = WebhookSender.from_adcp_legacy_hmac( |
| 47 | + secret=b"shared-secret-bytes", |
| 48 | + key_id="kid_buyer_42", |
| 49 | +) |
| 50 | +async with sender: |
| 51 | + result = await sender.send_mcp(url=..., task_id=..., status=...) |
| 52 | +``` |
| 53 | + |
| 54 | +Wire format matches `verify_webhook_hmac` in `adcp.signing.webhook_hmac`: `X-AdCP-Signature: sha256=<hex>` over `f"{timestamp}.{body}"`, with `X-AdCP-Timestamp` set fresh on every delivery (resends produce a new signature over the same body bytes — receivers enforcing a 300s skew window won't reject the retry). |
| 55 | + |
| 56 | +Plan to migrate to JWK signing before AdCP 4.0; the legacy `authentication` field is removed in 4.0. |
| 57 | + |
| 58 | +### Standard Webhooks v1 — Svix / Resend / standardwebhooks.com interop |
| 59 | + |
| 60 | +```python |
| 61 | +sender = WebhookSender.from_standard_webhooks_secret( |
| 62 | + "whsec_<base64-distributed-by-buyer>", |
| 63 | + key_id="kid_svix_42", |
| 64 | +) |
| 65 | +async with sender: |
| 66 | + result = await sender.send_mcp(url=..., task_id=..., status=...) |
| 67 | +``` |
| 68 | + |
| 69 | +The constructor takes the canonical `whsec_<base64>` form Svix and Resend distribute and base64-decodes it internally. **Do not pass the literal `whsec_…` string to `from_adcp_legacy_hmac`** — the AdCP-legacy scheme HMACs against raw bytes, so you'd silently produce signatures Svix would reject. The two constructors enforce different secret-encoding contracts at the type level so this swap can't happen. |
| 70 | + |
| 71 | +Wire format per spec: `webhook-id` / `webhook-timestamp` / `webhook-signature: v1,<base64>` over `f"{webhook_id}.{webhook_timestamp}.{body}"`. Each delivery gets a fresh `webhook-id` so a Svix-style receiver caching ids for replay defense doesn't false-positive on a legitimate retry. |
| 72 | + |
| 73 | +--- |
| 74 | + |
| 75 | +## Translation table |
| 76 | + |
| 77 | +### Pattern: hand-built bearer POST |
| 78 | + |
| 79 | +```python |
| 80 | +# Legacy |
| 81 | +import requests, json |
| 82 | +requests.post( |
| 83 | + url, |
| 84 | + data=json.dumps(payload), |
| 85 | + headers={ |
| 86 | + "Authorization": f"Bearer {token}", |
| 87 | + "Content-Type": "application/json", |
| 88 | + }, |
| 89 | +) |
| 90 | +``` |
| 91 | + |
| 92 | +```python |
| 93 | +# Equivalent |
| 94 | +sender = WebhookSender.from_bearer_token(token) |
| 95 | +async with sender: |
| 96 | + await sender.send_raw( |
| 97 | + url=url, |
| 98 | + idempotency_key=payload["idempotency_key"], |
| 99 | + payload=payload, |
| 100 | + ) |
| 101 | +``` |
| 102 | + |
| 103 | +`send_raw` requires `idempotency_key` as a kwarg — it's injected into the payload before serialization. Pass the same value you used to use; if your legacy code generated it ad hoc, call `generate_webhook_idempotency_key()` once and pass it in. |
| 104 | + |
| 105 | +### Pattern: hand-built HMAC POST with the sign-vs-body bug |
| 106 | + |
| 107 | +```python |
| 108 | +# Legacy — note the lurking serialization mismatch |
| 109 | +import hmac, hashlib, json, requests |
| 110 | +body_str = json.dumps(payload, separators=(",", ":")) |
| 111 | +sig = hmac.new(secret, body_str.encode(), hashlib.sha256).hexdigest() |
| 112 | +requests.post( |
| 113 | + url, |
| 114 | + json=payload, # ← reserializes with default separators; signature mismatches |
| 115 | + headers={"X-AdCP-Signature": f"sha256={sig}", ...}, |
| 116 | +) |
| 117 | +``` |
| 118 | + |
| 119 | +```python |
| 120 | +# Equivalent — sender signs and sends the same bytes |
| 121 | +sender = WebhookSender.from_adcp_legacy_hmac(secret, key_id="kid_buyer_42") |
| 122 | +async with sender: |
| 123 | + await sender.send_raw( |
| 124 | + url=url, |
| 125 | + idempotency_key=payload["idempotency_key"], |
| 126 | + payload=payload, |
| 127 | + ) |
| 128 | +``` |
| 129 | + |
| 130 | +The "sign one byte sequence, POST a different one via `json=`" bug is one of the most common HMAC-webhook integration failures in the field. `WebhookSender` serializes once with `json.dumps(...)` and POSTs the same bytes via `httpx.AsyncClient.post(content=body)` — the bug class is impossible. |
| 131 | + |
| 132 | +### Pattern: hand-built Standard-Webhooks (Svix-style) POST |
| 133 | + |
| 134 | +```python |
| 135 | +# Legacy |
| 136 | +import base64, hmac, hashlib, json, time, requests, uuid |
| 137 | +secret_bytes = base64.b64decode(secret_str.removeprefix("whsec_") + "==") |
| 138 | +msg_id = f"msg_{uuid.uuid4().hex}" |
| 139 | +ts = str(int(time.time())) |
| 140 | +body = json.dumps(payload, separators=(",", ":")).encode() |
| 141 | +mac = hmac.new(secret_bytes, f"{msg_id}.{ts}.".encode() + body, hashlib.sha256).digest() |
| 142 | +requests.post( |
| 143 | + url, |
| 144 | + data=body, |
| 145 | + headers={ |
| 146 | + "webhook-id": msg_id, |
| 147 | + "webhook-timestamp": ts, |
| 148 | + "webhook-signature": f"v1,{base64.b64encode(mac).decode()}", |
| 149 | + "Content-Type": "application/json", |
| 150 | + }, |
| 151 | +) |
| 152 | +``` |
| 153 | + |
| 154 | +```python |
| 155 | +# Equivalent |
| 156 | +sender = WebhookSender.from_standard_webhooks_secret(secret_str, key_id="kid") |
| 157 | +async with sender: |
| 158 | + await sender.send_raw( |
| 159 | + url=url, |
| 160 | + idempotency_key=payload["idempotency_key"], |
| 161 | + payload=payload, |
| 162 | + ) |
| 163 | +``` |
| 164 | + |
| 165 | +### Pattern: custom Docker localhost rewrite |
| 166 | + |
| 167 | +```python |
| 168 | +# Legacy — homemade rewrite for "deliver from Docker container to host webhook" |
| 169 | +url = url.replace("localhost", "host.docker.internal") |
| 170 | +url = url.replace("127.0.0.1", "host.docker.internal") |
| 171 | +``` |
| 172 | + |
| 173 | +```python |
| 174 | +# Equivalent — TransportHook composes with everything else |
| 175 | +from adcp.webhook_sender import WebhookSender |
| 176 | +from adcp.webhook_transport_hooks import DockerLocalhostRewrite |
| 177 | + |
| 178 | +sender = WebhookSender.from_jwk( |
| 179 | + jwk, |
| 180 | + transport_hooks=(DockerLocalhostRewrite(),), |
| 181 | + allow_private_destinations=True, # required — see note below |
| 182 | +) |
| 183 | +``` |
| 184 | + |
| 185 | +`DockerLocalhostRewrite` requires `allow_private_destinations=True` at sender construction — the rewrite produces a private-IP destination, and SSRF would reject the rewritten URL otherwise. The flag is the operator's explicit opt-in; it raises at construction time if forgotten so the misconfiguration doesn't bite at first delivery. |
| 186 | + |
| 187 | +For Linux containers without `--add-host=host.docker.internal:host-gateway`, pass `DockerLocalhostRewrite(rewrite_to="172.17.0.1")` (Docker's default bridge gateway). |
| 188 | + |
| 189 | +### Pattern: per-call retry loop with backoff |
| 190 | + |
| 191 | +```python |
| 192 | +# Legacy |
| 193 | +import time |
| 194 | +for attempt in range(5): |
| 195 | + try: |
| 196 | + response = requests.post(url, ...) |
| 197 | + if response.status_code < 500: |
| 198 | + break |
| 199 | + except requests.RequestException: |
| 200 | + pass |
| 201 | + time.sleep(2 ** attempt) |
| 202 | +``` |
| 203 | + |
| 204 | +```python |
| 205 | +# Equivalent — WebhookDeliverySupervisor owns retry + circuit breaker + dedup |
| 206 | +from adcp.webhooks import ( |
| 207 | + InMemoryWebhookDeliverySupervisor, |
| 208 | + WebhookDeliveryRequest, |
| 209 | +) |
| 210 | + |
| 211 | +supervisor = InMemoryWebhookDeliverySupervisor( |
| 212 | + sender=sender, |
| 213 | + # tune retry policy / circuit-breaker thresholds at construction |
| 214 | +) |
| 215 | +await supervisor.deliver(WebhookDeliveryRequest(url=..., payload=..., ...)) |
| 216 | +``` |
| 217 | + |
| 218 | +The supervisor handles retry policy, circuit-breaker state per buyer, and durable queueing. For Postgres-backed deployments, swap in `PgWebhookDeliverySupervisor` with the same Protocol — the calling code doesn't change. |
| 219 | + |
| 220 | +### Pattern: per-call SSRF check |
| 221 | + |
| 222 | +```python |
| 223 | +# Legacy |
| 224 | +host = urlparse(url).hostname |
| 225 | +ip = socket.gethostbyname(host) |
| 226 | +if ipaddress.ip_address(ip).is_private: |
| 227 | + raise ValueError("private IP not allowed") |
| 228 | +requests.post(url, ...) # ← TOCTOU: DNS may rebind between check and connect |
| 229 | +``` |
| 230 | + |
| 231 | +```python |
| 232 | +# Equivalent — automatic. WebhookSender pins the validated IP into the |
| 233 | +# transport so DNS rebinding cannot swap the connect target between |
| 234 | +# validation and POST. |
| 235 | +sender = WebhookSender.from_jwk(jwk) # SSRF runs on every send |
| 236 | +``` |
| 237 | + |
| 238 | +The owned-client path rebuilds an `AsyncIpPinnedTransport` per request, runs the full SSRF range check (loopback / RFC 1918 / link-local / CGNAT / IPv6 ULA / multicast / cloud metadata), enforces an optional port allowlist, and pins the connection to the validated IP. There is no "remembered to call the SSRF helper" failure mode. |
| 239 | + |
| 240 | +If your infrastructure uses a vetted egress proxy with mTLS to a fixed buyer set, pass your own `client=httpx.AsyncClient(...)` and the sender will trust the operator's transport instead. |
| 241 | + |
| 242 | +### Pattern: ad-hoc retry that re-signs with the original timestamp |
| 243 | + |
| 244 | +```python |
| 245 | +# Legacy — replays the original signature, which receivers reject after |
| 246 | +# the 300s skew window. |
| 247 | +saved = (signed_headers, body_bytes) |
| 248 | +time.sleep(60) |
| 249 | +requests.post(url, data=saved[1], headers=saved[0]) |
| 250 | +``` |
| 251 | + |
| 252 | +```python |
| 253 | +# Equivalent — WebhookSender.resend re-signs the same bytes with a |
| 254 | +# fresh signature on every retry, preserving idempotency_key for dedup. |
| 255 | +result = await sender.send_mcp(url=..., task_id=..., status=...) |
| 256 | +if not result.ok: |
| 257 | + retry = await sender.resend(result) |
| 258 | +``` |
| 259 | + |
| 260 | +`resend` works in every auth mode. JWK regenerates the 9421 Signature/Signature-Input headers; HMAC modes generate a new timestamp + signature over the same body; bearer mode is a no-op (the auth header is timestamp-independent). |
| 261 | + |
| 262 | +--- |
| 263 | + |
| 264 | +## Failure modes the new API closes |
| 265 | + |
| 266 | +* **Sign-vs-body divergence** — the sender owns marshaling; signed bytes equal sent bytes. |
| 267 | +* **Retry stales out the signature** — `resend` re-signs. |
| 268 | +* **Forgotten `idempotency_key`** — required kwarg on `send_raw`; receivers dedupe on it. |
| 269 | +* **DNS rebinding between SSRF check and POST** — automatic IP-pin on the owned-client path. |
| 270 | +* **Standard Webhooks secret encoding mistake** — typed split between `from_adcp_legacy_hmac(bytes)` and `from_standard_webhooks_secret(str)`. |
| 271 | +* **Docker rewrite without operator opt-in** — `DockerLocalhostRewrite` raises at construction unless `allow_private_destinations=True`. |
| 272 | +* **Hooks bypassing SSRF** — hooks run before SSRF, but SSRF validates the post-rewrite URL; the boundary holds. |
| 273 | + |
| 274 | +If your current sender doesn't have one of these properties, this is what you're trading up to. |
0 commit comments