Skip to content

feat: seal MCP event subscriptions into an envelope - #132

Open
ChiragAgg5k wants to merge 6 commits into
feat/events-protocolfrom
feat/events-envelope
Open

ChiragAgg5k wants to merge 6 commits into
feat/events-protocolfrom
feat/events-envelope

Conversation

@ChiragAgg5k

@ChiragAgg5k ChiragAgg5k commented Oct 9, 2026 •

Copy link
Copy Markdown
Member

Stack

feat/events ← #131 ← #132 ← #133 ← #134 ← #135 (events/subscribe / events/unsubscribe)

Merges into feat/events; feat/events → main lands as one feature after appwrite/appwrite#14293. Each PR's diff shows only its own layer, and every layer passes the full checklist on its own.

Summary

Part 3 of 4 for MCP Events (#127). Adds mcp_server_appwrite.events.envelope, the sealed subscription envelope that lets the hosted server stay stateless: each subscription is one Appwrite project webhook, and everything the ingress needs to deliver an event is sealed into that webhook's authPassword, which Appwrite sends back as HTTP Basic auth on every delivery.

The envelope format is not ours: it is the mcp-subscription+jwe encrypted envelope profile of mcp-event-subscriptions (mcp-event-subscriptions 0.1.0a1 on PyPI), sealing a Subscription Record v1. envelope.py is the thin adapter around it.

This is a self-contained library. Nothing is wired into server.py or http_app.py; the subscribe and ingress PRs consume it.

Blocked from end-to-end use by appwrite/appwrite#14293. Today authPassword fails above ~99 chars (encrypted column sized for plaintext), and its API validator is Text(256). Every envelope this produces is longer than both (see the size table). The state lives only in authPassword: nothing is split into authUsername or the URL.

What's in it:

  • Subscription frozen dataclass, the typed view of the record: id, project, name, arguments (sorted, read-only mapping), callback, secret (one whsec_ secret, kept out of repr), expires and refresh (epoch ms, written as the record's lease.expiresAt / lease.refreshBefore), principal (digest, never raw tokens). Subscription.record builds the record, Subscription.from_record reads it back and refuses one whose id does not hash from its contents.
  • Context(server, tenant, resource): what an envelope is bound to.
  • Keyring (from MCP_EVENTS_SEALING_KEYS) with seal(subscription, server), open(envelope, context) and record(envelope, context) (both take an optional now that only fixtures pin), plus key_id(envelope) to read the kid from the JWE header before opening.
  • Errors: EnvelopeError with an EnvelopeFailure reason (malformed, unknown_key, authentication_failed, binding_mismatch), SubscriptionExpired (kept apart so the ingress can answer "expired" with a 2xx drop and "invalid" with 401), EnvelopeTooLarge, InvalidSubscription, KeyringError.
  • Principal(issuer, subject, client).digest: base64url of the first 128 bits of SHA-256 over canonical JSON of OAuth iss, sub, client_id (22 chars).
  • subscription_id(principal, callback, name, arguments): sub_ + 32 hex chars of SHA-256 over canonical JSON. Exactly 36 chars, valid for Appwrite's CustomId/Key validator. Argument order does not change it. valid_subscription_id accepts exactly sub_[0-9a-f]{32}.
  • Derived Appwrite webhook secret (Keyring.signing_key) and appwrite_signature / verify_appwrite_signature for X-Appwrite-Webhook-Signature.
  • Dependencies: mcp-event-subscriptions>=0.1.0a1,<0.2 (brings jwcrypto 1.6.1; jsonschema 4.26.0 was already locked). cryptography stays a direct dependency because the signing key derivation imports its HKDF.

Format

JWE Compact, exactly as the profile specifies:

base64url(protected header)..base64url(iv).base64url(ciphertext).base64url(tag)
  • Protected header {"alg":"dir","enc":"A256GCM","kid":"<key id>","typ":"mcp-subscription+jwe","cty":"application/json"}, the empty encrypted-key segment of dir, a random 96-bit IV per seal. The sealing key is the A256GCM key.
  • Plaintext {"profile":"mcp-subscription-envelope-v1","context":{"server","tenant","resource"},"subscription":<record>}. The record holds version: 1, id, principal, event.{name,arguments}, delivery.{mode:"webhook",url,secret}, lease.{expiresAt,refreshBefore} (UTC RFC 3339 with milliseconds and Z) and extensions: {}.
  • Context binding: server is the stable deployment id from MCP_PUBLIC_URL, tenant the project id, resource the subscription id (which is also the webhook id). When opening, the expected context comes only from trusted configuration, the routed webhook id and X-Appwrite-Webhook-Project-Id, never from the token. An envelope copied onto another webhook, project or deployment does not open. On top of the profile, the record must hash back to the subscription id (binding_mismatch otherwise).
  • The profile checks expiry when opening: an authentic, correctly bound but expired envelope raises SubscriptionExpired.
  • The envelope alphabet is [A-Za-z0-9._-], safe as a Basic auth password. Appwrite only sends Basic auth when authUsername is non-empty too, so the subscribe PR sets a fixed, non-secret username.

Example (throwaway key; tablesdb.row.created, 59-char callback URL, one 32-byte secret):

id:       sub_d9c402c938a2c222018129fb01a3a9fc
record:   {"delivery":{"mode":"webhook","secret":"whsec_ruYQcd+PH1IwgzTqgyCc/vi8cr9ixFmryDML9P1VLGU=","url":"https://chatgpt.com/backend-api/mcp/events/webhook/3f2c9a7e"},"event":{"arguments":{"database_id":"main","project_id":"6630f1a2b3c4d5e6f7a8","table_id":"support_tickets"},"name":"tablesdb.row.created"},"extensions":{},"id":"sub_d9c402c938a2c222018129fb01a3a9fc","lease":{"expiresAt":"2025-10-09T09:53:20.000Z","refreshBefore":"2025-10-09T09:47:20.000Z"},"principal":"8izYeiSqdyEyFfJntvQ9aA","version":1}
context:  {"server":"https://mcp.appwrite.io","tenant":"6630f1a2b3c4d5e6f7a8","resource":"sub_d9c402c938a2c222018129fb01a3a9fc"}
header:   {"alg":"dir","cty":"application/json","enc":"A256GCM","kid":"k1","typ":"mcp-subscription+jwe"}
envelope: eyJhbGciOiJkaXIiLCJjdHkiOiJhcHBsaWNhdGlvbi9qc29uIiwiZW5jIjoiQTI1NkdDTSIsImtpZCI6ImsxIiwidHlwIjoibWNwLXN1YnNjcmlwdGlvbitqd2UifQ..frHVZrJOoN_e2mFt.USLRUBEB…LRiEgqlTKMCV-D8kuoqSzg   (1087 chars)
secret:   85b8799546adde3d307b5d09b53526f2f90b3097de8f418ae23025594a585c6c

Size budget

Measured with 36-char Appwrite ids, a 22-char principal digest, key id k1 and server https://mcp.appwrite.io. Secrets are whsec_ + base64 of the stated byte length.

Case Envelope length
tablesdb.row.created, 59-char URL, one 32-byte secret 1108
users.user.created, 100-char URL, one 32-byte secret 1115
tablesdb.row.created, 150-char URL, one 32-byte secret 1322
functions.deployment.completed + status, 300-char URL, one 64-byte secret 1551
Worst case: tablesdb.row.created, 300-char URL, one 64-byte secret 1580
ENVELOPE_BUDGET (seal raises EnvelopeTooLarge above it) 2048
Appwrite authPassword today: reliable limit (encrypted column) ~99 (fails randomly from ~90)
Appwrite authPassword today: API validator Text(256) 256

The JWE envelope is roughly 550 to 680 chars longer than the previous custom format (base64url header, JSON wrapper, readable record keys, context), and still about 470 chars under the budget in the worst case. What appwrite/appwrite#14293 must allow is unchanged: authPassword of at least ENVELOPE_BUDGET = 2048 characters, stored intact, i.e. the column change already proposed there (httpPass → VAR_TEXT 65535) and raising the authPassword param validator on create and update from Text(256) to at least Text(2048). Subscribe maps EnvelopeTooLarge to -32602 (callback URL too long).

Key management

  • MCP_EVENTS_SEALING_KEYS=<id>:<base64 of 32 random bytes>[,<id>:<base64 of 32 random bytes>...], read from the environment like the other hosted config (Keyring.from_env()), documented in .env.example and docs/development.md. In production it comes from Parameter Store.
  • Each key is 32 random bytes, as the profile requires, and is used directly as the JWE key, so any implementation of the profile holding the same key opens our envelopes. Keys are validated at startup: exactly 32 bytes (standard or URL-safe base64, padding optional) with at least 16 distinct byte values, which rejects zero, repeated and hand-typed keys. Key ids follow the profile's kid syntax ([A-Za-z0-9_-]{1,128}) and must be unique. A missing variable raises KeyringError with a generation hint (openssl rand -base64 32).
  • The first key seals, every key opens; opening uses the kid named in the header (no trial decryption).
  • Rotation is unchanged: put the new key first; drop the old one after the longest subscription TTL has passed. Subscriptions refresh before they expire, so each refresh re-seals with the new key. (feat: serve events/subscribe and events/unsubscribe #135 stages this across replicas as three deploys.)
  • The Appwrite webhook secret is hex(HMAC-SHA256(HKDF-SHA256(sealing key, "mcp-events/appwrite-signature/v1"), subscription id)): 64 chars, within Appwrite's Text(256, 8). The ingress derives it from the ring, so no storage is needed.
  • Appwrite signs deliveries as base64(HMAC-SHA1(url . body, secret)) (src/Appwrite/Platform/Workers/Webhooks.php), where url is the webhook's configured URL. The ingress must rebuild the canonical ingress URL, not trust the URL the request arrived on.

Tests

Envelope behavior is covered end to end by the ingress flows in #134 (tests/e2e/test_events_ingress.py), which run against the real server:

  • Seal/open round trip: every delivered event passes through Keyring.seal and the ingress's open.
  • Tampering and binding (401 envelope): a flipped envelope byte; an envelope for another project; one copied from another webhook; one sealed with the server's key for the same webhook and project but by another deployment (another MCP_PUBLIC_URL); one with a header that names a ring key but not the profile's algorithm pair; one sealed with the server's own key for this webhook and project but holding another subscription's contents (binding_mismatch).
  • Malformed envelopes (401 credentials): the old v1.k1.… format, too many parts, a non-empty encrypted-key segment, a header that is not base64url JSON, a header without kid, an invalid kid. Foreign path ids such as sub_zzzz… or sub_AAAA… also get 401 credentials.
  • Key rotation by restarting the server: k1 delivers; restarted with k2,k1, the old envelope still delivers and a k2 one does too; restarted with k2 only, the k1 envelope gets 200 retired_key and nothing is delivered.
  • Expiry: an expired subscription gets 200 dropped expired and nothing is delivered.
  • Keyring parsing: the real entry point exits when MCP_EVENTS_SEALING_KEYS is missing, and short, all-zero, repeated, non-base64, id-less and duplicate-id keys stop startup with KeyringError.

Unit tests kept (tests/unit/test_events_envelope.py, 11 tests), and why they are not e2e:

  • Profile conformance vector (new, the only new unit test). The package's published test-vectors/encryption/jwe-v1.json, copied unchanged to tests/unit/fixtures/jwe-v1.json (the vector ships in the GitHub repo, not the wheel). Our keyring parses its key, Keyring.record opens its token (clock pinned to 2029, since the vector's lease ends in 2030) to exactly the published record, Keyring.open refuses it only for binding_mismatch (its sub_example id is not one subscription_id() derives), and the published record maps field for field onto our Subscription, whose repr does not show the secret. Every e2e envelope is sealed and opened by this same server, so only this vector ties our format to the profile.
  • Appwrite signature vector produced with PHP 8.5, exactly as the worker computes it, plus rejection of a changed URL, body, key or signature. This is external truth: the e2e harness signs deliveries with its own Python implementation of the same formula, so only this vector ties both to PHP.
    php -r '$url="https://mcp.appwrite.io/appwrite/webhooks/sub_0123456789abcdef0123456789abcdef"; $payload="{\"\$id\":\"6630f1a2b3c4d5e6f7a8\",\"status\":\"failed\"}"; $key="appwrite-signing-key-for-tests"; echo base64_encode(hash_hmac("sha1", $url . $payload, $key, true));'
    # SgM0XNLzyHAwzlgQA2iZykDQI8M=
  • Subscription id determinism and charset: same inputs give the same id, argument order doesn't matter, every input changes it, 200 generated ids satisfy Appwrite's CustomId rules. Foreign ids are refused, including Appwrite-valid ones that are not 32 lowercase hex characters. Also covered: the principal digest is stable and 22 chars, and the derived webhook secret fits Appwrite's Text(256, 8). Nothing derives an id from a request until events/subscribe (PR 5), which replaces these with its e2e flow.
  • Size budget (one test): the realistic worst case (300-char URL, 64-byte secret, longest arguments) fits ENVELOPE_BUDGET, and a 2048-char URL raises EnvelopeTooLarge. This is the contract Encrypted and length-limited attributes return 500 instead of 400 (webhooks authPassword/authUsername/url, variables value) appwrite#14293 has to meet.

Review changes

  • d14a220 (Hansi): valid_subscription_id accepted any 32-character Appwrite-safe suffix. It now accepts only sub_ followed by 32 lowercase hex characters, which is what subscription_id() emits.

  • 7ede499: the custom v1.<kid>.<base64url(nonce | ciphertext | tag)> AES-GCM format is replaced by the mcp-subscription+jwe profile from mcp-event-subscriptions. Deleted: the AES-GCM/HKDF seal key, the one-letter plaintext, the custom associated data and the envelope parser, and the per-case size tests (one budget test stays). The record holds one delivery.secret instead of a secrets tuple, matching the no-dual-sign behaviour of subscribe. Binding moved from AES-GCM associated data to the profile's encrypted context, which now also binds the server. Adds the published-vector test above.

  • d4d5931, 350cea4 (Hansi): the vector's lease ends in 2030, so Keyring.record / Keyring.open take an optional now and the vector test pins it, as the package's own tests do. A Subscription's secret is kept out of its repr (same finding as Callback in feat: sign and deliver MCP events safely #133).

Verification

Run locally on Python 3.12.8, lockfile written with uv 0.11.22 (the CI version):

  • uv run --group dev ruff check src tests: pass
  • uv run --group dev black --check src tests: pass
  • uv run --group dev pyright: 0 errors
  • uv run python -m unittest discover -s tests/unit: 279 tests OK
  • uv run --group e2e python -m unittest discover -s tests/e2e: 2 tests OK (about 3 s)
  • docker build -t appwrite-mcp:jwe .: builds

@hansi-codes

hansi-codes Bot commented Oct 9, 2026 •

Copy link
Copy Markdown

🟢 Tier S · Ready to merge

This pull request adds the hosted MCP Events catalog and protocol support, plus stateless subscription envelopes, key management, and Appwrite webhook-signature helpers. The newest commits adapt the envelope implementation to the mcp-subscription+jwe profile and add its published conformance vector.

Latest changes: The latest commits replace the custom AES-GCM envelope format with the external JWE profile, adapt subscriptions and context binding to its record format, and add a conformance-vector test and dependency.

Verdict New comments Fixed Still open
✅ Approved 0 0 0
📂 Walkthrough · 10
File Change
README.md, docs/events.md, docs/flags.md, AGENTS.md Document MCP Events, its hosted behavior, and the testing workflow.
.github/workflows/ci.yml Add CI coverage for the hosted end-to-end event flows.
.env.example, docs/development.md Document event sealing-key configuration and the updated JWE envelope profile.
pyproject.toml, uv.lock Add the event-subscription profile package and lock its dependencies.
src/mcp_server_appwrite/events/__init__.py, catalog.py, errors.py, protocol.py Add the event catalog, protocol methods, feature gating, and event-specific errors.
src/mcp_server_appwrite/events/envelope.py Add subscription identity and sealing, key rotation, context binding, and Appwrite signature helpers; the latest commits adapt these to the JWE profile.
src/mcp_server_appwrite/flags.py, src/mcp_server_appwrite/server.py Add the Events flag and register the event protocol for hosted HTTP servers.
tests/e2e/support.py, tests/e2e/test_events_protocol.py Add a real-HTTP test harness and hosted event protocol coverage.
tests/unit/test_events_catalog.py, tests/unit/test_events_envelope.py, tests/unit/test_events_protocol.py Test catalog validation, subscription identifiers, signature behavior, envelope sizing and profile conformance, and stdio gating.
tests/unit/fixtures/jwe-v1.json Add the published JWE conformance vector used by the envelope adapter test.

Reviewed the commits since d14a220 · Details · Comment @hansi-codes review to re-run, or mention @hansi-codes with a question.

@hansi-codes hansi-codes Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Tier A · See the inline comments. Summary

Comment thread src/mcp_server_appwrite/events/envelope.py Outdated
Each MCP Events subscription is stored only in its Appwrite webhook. The
envelope seals the delivery state into authPassword with AES-256-GCM, bound
to the subscription id and project, and the webhook signing key is derived
so the ingress can verify deliveries without any storage.
Round trips, tampering, binding, rotation and expiry are covered through
the ingress e2e flows in the layer above. What stays is the PHP-computed
Appwrite signature vector, subscription id determinism and charset until
subscribe has an e2e flow, and the size budget for appwrite/appwrite#14293.
@ChiragAgg5k
ChiragAgg5k force-pushed the feat/events-envelope branch from aef5109 to a995f8f Compare October 9, 2026 13:53
@ChiragAgg5k
ChiragAgg5k changed the base branch from main to feat/events-protocol October 9, 2026 13:53

@hansi-codes hansi-codes Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Tier A · See the inline comments. Summary

Comment thread src/mcp_server_appwrite/events/envelope.py Outdated
valid_subscription_id accepted any 32-char Appwrite-safe suffix, so ids like
sub_zzzz... passed as ours although subscription_id() only ever emits 32
lowercase hex characters. The ingress route and managed-webhook recognition
use it to tell our ids from foreign ones, so they now reject anything else.

@hansi-codes hansi-codes Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Tier A · Looks good to merge. Summary

…file

The custom v1.<kid>.<AES-GCM> envelope becomes a Subscription Record v1
sealed with the package's mcp-subscription+jwe profile, bound to this
server, the project and the webhook id. The adapter keeps what the profile
leaves to us: subscription ids, the principal digest, the derived Appwrite
signing key and the authPassword size budget.

@hansi-codes hansi-codes Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Tier A · Looks good to merge. Summary

Comment thread tests/unit/test_events_envelope.py Outdated
… repr

The published vector's lease ends in 2030, so Keyring.record and open take
an optional clock that the vector test pins, as the package's own tests do.
A Subscription's whsec_ secret is no longer part of its repr.

@hansi-codes hansi-codes Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Tier S · Looks good to merge. Summary

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant