feat(metrics): export HTTP request rate, error rate and p95 latency

[arandomogg]

Summary

Lens exports seven Prometheus metrics on a public /metrics route, but nothing for the HTTP layer. There is no request rate, no error rate — nothing counted 5xx at all — and no request latency, since db_query_duration_seconds covers the database rather than the request. price_requests_total counts price calls specifically, so it cannot answer "is the API healthy".

This adds two metrics on the existing registry, exported through the existing /metrics route:

http_requests_total{method, route, status_class}
http_request_duration_seconds{method, route}

closes #148

Getting the label set right

Cardinality is the actual work here, not the hook. The naive version labels a histogram by req.url, which mints a new time series for every asset pair, every address and every cursor Lens is asked about — an unbounded set driven by callers, which eventually degrades the Prometheus scraping it.

So:

• route is the matched Fastify route template (/price/:assetA/:assetB), taken from req.routeOptions.url, never the resolved URL.
• Unrouted requests collapse to the literal unmatched rather than the raw path, so a scanner probing a thousand URLs cannot mint a thousand series.
• status_class is 2xx/4xx/5xx, not the exact code. This bounds the series count at methods x routes x 5 instead of methods x routes x ~60. Exact status is easy to add later and painful to remove once dashboards depend on it.

Bucket choice

The buckets are chosen for this service and the reasoning is recorded in a comment beside the definition. prom-client's defaults cluster below 1s and suit fast in-process handlers; Lens handlers are not that. A request does a Redis cache lookup (single-digit ms) or, on a miss, a Postgres aggregate plus — for /price and /route — Horizon/Soroswap network round-trips.

[0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5, 10] gives resolution in two regions: 5-50ms for cache hits (enough to see the Redis path regress before users notice) and 100ms-2.5s for DB aggregates and upstream venue calls, which is where p95 actually lives and where an SLO would be set. The 5s and 10s buckets catch upstream stalls short of a timeout.

Catching errors, and registration order

The observation runs in onResponse, which fires for every response that is sent — including ones produced by the error handler, by reply.send() inside an onRequest hook (401 from auth, 402 from x402, 429 from the rate limiter) and by the default 404 handler. A 500 escaping the counter is the exact case this exists to catch.

Per the issue note, the plugin is registered first in src/index.ts, ahead of the network selector, API-key auth, the rate limiter and x402. All of those reject from onRequest, and Fastify runs onRequest hooks in registration order, so registering later would start the timer too late and lose exactly the rejected traffic an operator most wants to see.

/metrics itself is excluded from counting, or the scraper continuously inflates the numbers it is reading and request rate never falls to zero on an idle service.

price_requests_total is left alone — it means something different and dashboards may use it.

Verification

Full unit suite is green, and the branch adds 12 tests without touching the existing ones:

baseline (upstream/main):  37 files, 304 tests passed
this branch:               38 files, 316 tests passed

npx tsc --noEmit is clean.

Beyond the committed tests, I checked the rejection paths against the real plugins rather than only a stub, to confirm the registration-order claim actually holds:

# @fastify/rate-limit, max: 2, five requests
[ { value: 2, labels: { method: GET, route: /thing, status_class: 2xx } },
  { value: 3, labels: { method: GET, route: /thing, status_class: 4xx } } ]

# 402 sent from a preHandler hook (x402 shape)
[ { value: 1, labels: { method: GET, route: /price/:a/:b, status_class: 4xx } } ]

# DELETE against a GET-only path
[ { value: 1, labels: { method: DELETE, route: unmatched, status_class: 4xx } } ]

Note the x402 case keeps the route template rather than degrading to unmatched, because routing has already resolved by preHandler.

Acceptance criteria

• http_requests_total and http_request_duration_seconds exported on /metrics
• Labels use the route template; a test asserts 100 requests to distinct addresses produce one series, not 100 (and that the resolved URL never appears in a label)
• 5xx counted, asserted with a route that throws
• 4xx counted and distinguishable from 5xx
• Unmatched routes do not create a series per path — 25 distinct 404 paths produce one unmatched series
• Latency observed for both successful and failed requests
• Histogram buckets justified in a comment
• Docs section listing what is exported and what an alert should look like

Docs

docs/http-metrics.md carries the full label reference, the bucket rationale, PromQL for all three signals and suggested alerting rules. The README gains a short Observability section pointing at it.

One deliberate choice in the alerts: LensNoTraffic is a separate rule rather than folded into the error-rate alert, because the 5xx ratio is undefined when there is no traffic — a service serving zero requests is a real problem an error ratio will never fire on.

Files

• src/metrics.ts — the two new metric definitions
• src/middleware/httpMetrics.ts — the plugin, plus exported statusClass and routeLabel helpers so the labelling rules are unit-testable on their own
• src/index.ts — registration, first in the chain
• tests/httpMetrics.test.ts — 12 tests
• docs/http-metrics.md, README.md — documentation

[drips-wave]

@arandomogg Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

[Miracle656]

Approved and merging. Verified locally on a merge with main: tsc --noEmit clean, full suite 324 passed / 1 skipped.

The three signals are the standard ones; what makes this PR good is that it treats cardinality as the design constraint rather than an afterthought, and gets every one of the four decisions right:

• route is req.routeOptions.url, the template. Labelling by req.url is the single most common way a metrics PR takes down the Prometheus instance scraping it — every asset pair, address and cursor mints a fresh time series, and the set is driven by callers, so it has no ceiling. The test asserting that 100 requests to 100 distinct addresses produce one series is the right test, because this is a regression you cannot see in a dashboard until it is already expensive.
• Unmatched requests collapse to the literal unmatched. Without this the 404 path is a free cardinality bomb — a scanner probing a thousand URLs mints a thousand series, from outside your system.
• status_class, not status. Bounds the count at methods × routes × 5. And the note that it is easy to add a label later and painful to remove one dashboards already depend on is exactly the right way round to be conservative.
• Buckets chosen for this service, not copied from the defaults. prom-client's defaults assume fast in-process handlers; Lens has a bimodal latency profile — single-digit-ms Redis hits, then Postgres aggregates and Horizon/Soroswap round-trips. Giving resolution in both regions is the difference between a p95 you can set an SLO against and one that is always in the same bucket.

The registration-order argument is the part I checked hardest, and it holds. All of the network selector, API-key auth, the rate limiter and x402 reject from onRequest, and Fastify runs those in registration order — so a plugin registered after them never starts its timer for a rejected request. I confirmed in src/index.ts: metrics at line 73, auth at 87, rate limit at 98, x402 at 130. Correct, and the reason is written down where the next person to add a plugin will read it.

Pairing onRequest with onResponse rather than onSend or a route wrapper is the same instinct — onResponse fires for error-handler responses, for reply.send() from inside another onRequest hook, and for the default 404. A 500 that escapes the counter is precisely what this exists to catch.

Excluding /metrics from itself is a small thing that saves a confusing week: a scraper on a fixed interval otherwise inflates the numbers it is reading, and request rate never falls to zero on an idle service.

Also appreciated: leaving price_requests_total alone and saying so. Redefining an existing metric's meaning breaks whatever is already alerting on it, silently.

Thanks — this is a genuinely well-argued PR.