Add graph ql subscriptions for live prices - #76
Conversation
Attach response schemas to every route in src/api/rest.ts (/status, /price/:a/:b, /price/:a/:b/route, /price/:a/:b/history, /pools) and add shared schema objects in src/api/schemas.ts. In dev/test a validating serializer checks each outgoing payload against its schema and throws on a mismatch, so accidental response-shape changes fail loudly instead of shipping silently. Production keeps Fastify default (fast) serialization for the same schemas, so there is no runtime impact. Adds src/__tests__/schemaValidation.test.ts asserting a known endpoint matches its schema and that extra/wrong-typed fields are rejected. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Attach response schemas to every route in src/api/rest.ts (/status, /price/:a/:b, /price/:a/:b/route, /price/:a/:b/history, /pools) and add shared schema objects in src/api/schemas.ts. In dev/test a validating serializer checks each outgoing payload against its schema and throws on a mismatch, so accidental response-shape changes fail loudly instead of shipping silently. Production keeps Fastify default (fast) serialization for the same schemas, so there is no runtime impact. Adds src/__tests__/schemaValidation.test.ts asserting a known endpoint matches its schema and that extra/wrong-typed fields are rejected.
…ts' of https://github.com/mikkyvans0-source/Lens into Add-JSON-Schema-response-validation-to-all-REST-endpoints
|
@mikkyvans0-source Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits. You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀 |
Miracle656
left a comment
There was a problem hiding this comment.
The priceUpdated GraphQL subscription over the existing Mercurius endpoint is a clean approach (good call keeping the REST /ws channel untouched).
The blocker: this branch is stacked on your JSON-schema-validation work (#75), so it also re-includes schemas.ts / schemaValidation.test.ts / the rest.ts changes. #75 has now been merged to main, so please rebase on the latest main and drop the #75 files — leaving just the subscription work (events.ts, graphql.ts, the ingester publishPriceUpdate calls, and graphqlSubscription.test.ts). I'll merge once it's rebased.
|
Friendly nudge — the GraphQL subscriptions PR is mergeable but still has the earlier change requests outstanding. Please rebase on the latest |
|
Status update, and an apology — this has been open since June. It now conflicts with
That is more churn than is fair to ask you to absorb blind, so before you spend time rebasing, two things worth knowing:
If you still want it, rebase onto current Either way the delay here is ours. |
…ions-for-live-prices # Conflicts: # README.md # src/__tests__/schemaValidation.test.ts # src/api/rest.ts # src/api/schemas.ts
…ions-for-live-prices
…ions-for-live-prices
Resolved the merge with a much-advanced main: - src/api/schemas.ts, schemaValidation.test.ts and rest.ts landed separately on main; took main's versions, leaving this PR to the subscription work it is actually about. - README kept both the Observability section from Miracle656#152 and this PR's subscription guide. Then closed a correctness gap the merge created. PricePublishedEvent carried only { pair, price, ts }. That was complete when one process indexed one chain; since Miracle656#117 every enabled network runs its own ingester loop and all of them publish to this one emitter, so a subscriber to XLM/USDC would receive testnet and mainnet prices interleaved with nothing to tell them apart — a feed that looks noisy rather than wrong, which is the harder kind to notice. - PricePublishedEvent gains a `network` field, threaded from all three ingesters. snapshotPool took a network parameter to do it; every other call site already had one in scope. - The GraphQL PriceUpdate type exposes `network`, and priceUpdated takes an optional `network` argument that filters the stream. - The filter compares with `== null`, not `=== undefined`. A client that passes the variable explicitly sends null rather than omitting it, so `=== undefined` would have silently delivered nothing to every such client. Caught by the new test, not by review. - Two tests: one network's price must not reach a subscriber to the other, and omitting the argument delivers both, each tagged. tsc clean; full suite 393 passed / 1 skipped.
1bff563 to
9ce755d
Compare
Miracle656
left a comment
There was a problem hiding this comment.
Approved and merging — I rebased this onto a much-advanced main and pushed the result to your branch (9ce755d), since the wave has closed.
The subscription implementation is well judged. Three things in particular:
graphql-transport-ws, not subscriptions-transport-ws. The legacy protocol is deprecated and unmaintained, and Mercurius still defaults to it — so this was a deliberate choice rather than an accepted default. It means any modern graphql-ws client works out of the box.
One app-level bridge listener, not one per subscriber. Attaching a priceEmitter listener inside subscribe would add a listener per connected client, blow past Node's maxListeners warning at eleven subscribers, and leak one per dropped connection. Bridging once onto a pubsub topic and letting withFilter fan out is the right shape.
The afterEach asserting priceEmitter.listenerCount(...) === 0 after app.close(). That is a leak test, and it is the test most people skip. It is also what told me the onClose hook actually detaches — I did not have to take it on trust.
What I changed, and why it mattered:
The merge was the easy half — src/api/schemas.ts, schemaValidation.test.ts and rest.ts had landed separately on main, so I took main's versions and left this PR to the subscription work it is actually about. The README kept both its section and #152's.
The substantive part is that PricePublishedEvent carried no network. { pair, price, ts } was complete when one process indexed one chain. Since #117 landed, every enabled network runs its own ingester loop and all of them publish to this one emitter — so a subscriber to XLM/USDC on a dual-network deployment receives testnet and mainnet prices interleaved, with nothing on the message to tell them apart. That produces a feed that looks noisy rather than wrong, which is the harder kind of bug to notice and the harder one to explain afterwards.
So:
PricePublishedEventgainsnetwork, threaded from all three ingesters.snapshotPoolneeded a new parameter; every other call site already had one in scope.PriceUpdateexposesnetwork, andpriceUpdatedtakes an optionalnetworkargument that filters the stream.- Two tests: one network's price must not reach a subscriber to the other, and omitting the argument delivers both, each tagged with its own.
One bug the new tests caught that review would not have. My first filter read network === undefined. A client that passes the variable explicitly sends null, not undefined — so every such client would have matched nothing and received an open socket delivering silence forever. Exactly the failure mode your complete test was already guarding against from the other direction. It compares with == null now.
Worth noting the test file is not covered by tsc (include is src/**/* but vitest runs untypechecked here), so the missing required field would not have surfaced as a compile error — the same trap that bit me on wraith #169.
Verified: tsc --noEmit clean, full suite 393 passed / 1 skipped.
The changeset was written before the network dimension was added in the #76 merge. It advertised priceUpdated(pair) emitting { pair, price, ts }, which would have gone into the 0.3.0 changelog as the public description of an API that also takes an optional network argument and emits a network field.
Summary
Closes #56
Added a priceUpdated(pair: String!) GraphQL subscription that streams live prices over the existing Mercurius/Fastify /graphql endpoint — no separate WebSocket stack.
Changes
src/events.ts — Added a minimal PRICE_PUBLISHED event with a { pair, price, ts } payload and a publishPriceUpdate() helper. Kept the existing priceEmitter/PRICE_UPDATE intact so the REST /ws channel is untouched.
src/api/graphql.ts —
Added a PriceUpdate type and a Subscription.priceUpdated(pair) field.
Enabled Mercurius subscriptions with the graphql-transport-ws subprotocol (fullWsTransport: true) so standard graphql-ws clients connect.
Resolver uses withFilter to forward only the requested pair.
A single app-level bridge listener forwards priceEmitter events onto app.graphql.pubsub, detached on onClose so the listener doesn't leak.
src/ingesters/sdex.ts, amm.ts, soroswap.ts — Each ingester calls publishPriceUpdate({ pair, price, ts }) after recording a new price, alongside the existing webhook dispatch.
README.md — Documented the subscription with a graphql-ws client snippet showing subscribe and the unsubscribe (clean-close) call.
src/tests/graphqlSubscription.test.ts — Integration test over a real ws connection covering all three acceptance criteria: streaming for the subscribed pair, server-side filtering of other pairs, and clean unsubscribe via complete. Also asserts the bridge listener is detached on shutdown.
Verification
Tests: full suite passes — 85/85, including the 3 new subscription tests.
Typecheck: no new errors. The two webhookDispatcher.ts errors are pre-existing (Prisma client isn't generated locally) and unrelated — I confirmed they exist on the unmodified tree via git stash.