store.go's pin budget is shared by every pinned namespace, and store.go:100 already warns that they compete for it. #234 added two more — BilledPrefix (cg:bin:) and UsagePrefix (cg:use:) — beside the existing SumPrefix (cg:sum:), all three session-keyed.
cg:bin: is written on every response, not only on summarizing sessions. So the number of pinned entries now scales with the number of sessions the proxy has seen, not with the number of sessions any component is managing state for.
A review measured the consequence on a synthetic load: pinnedN == pinCap exactly — 2,500 at the shipped defaults, from 20,000 sessions × 3 keys held at max.
Why saturation matters
Once the budget is full, admission silently declines. The entries that then go unpinned are whichever are written next — and that includes cg:frozen: and cg:result:, which are cache-destructive to lose: an evicted freeze record means the next turn's decision diverges from the cached prefix and forces a full-suffix cache write. That is the exact loss pinning exists to prevent, and it lands on a namespace that did nothing wrong.
The failure is also invisible. No metric exposes saturation. pinnedN and pinCap are internal; nothing on the dashboard or in /stats says "the pin budget is full and admissions are being refused", so the first symptom is a cache-hit-rate regression with no proximate cause.
What is worth deciding
-
A metric first, whatever else changes. pinned_entries and pin_cap in /stats, and a warning log the first time an admission is refused. Saturation should not be something a reviewer has to construct a load test to discover. This is small and it is the part that should not wait.
-
Whether cg:bin: needs to be pinned at all. It holds the previous response's billed input, read by Trigger.FracResolvable and the fill gate. Losing it makes the fill gate un-evaluable for one turn, which FracResolvable already handles by declining rather than guessing — a forgone firing opportunity, not a wrong number and not a cache write. Compare cg:sum:, where loss means re-sending a full transcript, and cg:seen:, where loss makes a warm prefix read cold. cg:bin: looks like the cheapest of the pinned namespaces to lose, which suggests it is the one that should not be holding a slot.
-
Whether the budget should be per-namespace rather than global. A global budget means the namespace with the highest write rate crowds out the ones with the highest loss cost, which is backwards. Per-namespace reserves would make the tradeoff explicit, at the cost of a more complicated admission rule.
-
Whether the cap should scale with the store's configured size rather than being a fixed 2,500.
Scope
This is a store concern, not a summarize one, and it predates #234 in mechanism — that PR raised the write rate against an existing shared budget rather than introducing the sharing. Item 1 is independent of the rest and could ship on its own.
Found by the review on #234, which went looking for a memory leak in the new namespaces, correctly reported that there is none (the TTL still applies), and found this instead.
store.go's pin budget is shared by every pinned namespace, andstore.go:100already warns that they compete for it. #234 added two more —BilledPrefix(cg:bin:) andUsagePrefix(cg:use:) — beside the existingSumPrefix(cg:sum:), all three session-keyed.cg:bin:is written on every response, not only on summarizing sessions. So the number of pinned entries now scales with the number of sessions the proxy has seen, not with the number of sessions any component is managing state for.A review measured the consequence on a synthetic load:
pinnedN == pinCapexactly — 2,500 at the shipped defaults, from 20,000 sessions × 3 keys held atmax.Why saturation matters
Once the budget is full, admission silently declines. The entries that then go unpinned are whichever are written next — and that includes
cg:frozen:andcg:result:, which are cache-destructive to lose: an evicted freeze record means the next turn's decision diverges from the cached prefix and forces a full-suffix cache write. That is the exact loss pinning exists to prevent, and it lands on a namespace that did nothing wrong.The failure is also invisible. No metric exposes saturation.
pinnedNandpinCapare internal; nothing on the dashboard or in/statssays "the pin budget is full and admissions are being refused", so the first symptom is a cache-hit-rate regression with no proximate cause.What is worth deciding
A metric first, whatever else changes.
pinned_entriesandpin_capin/stats, and a warning log the first time an admission is refused. Saturation should not be something a reviewer has to construct a load test to discover. This is small and it is the part that should not wait.Whether
cg:bin:needs to be pinned at all. It holds the previous response's billed input, read byTrigger.FracResolvableand the fill gate. Losing it makes the fill gate un-evaluable for one turn, whichFracResolvablealready handles by declining rather than guessing — a forgone firing opportunity, not a wrong number and not a cache write. Comparecg:sum:, where loss means re-sending a full transcript, andcg:seen:, where loss makes a warm prefix read cold.cg:bin:looks like the cheapest of the pinned namespaces to lose, which suggests it is the one that should not be holding a slot.Whether the budget should be per-namespace rather than global. A global budget means the namespace with the highest write rate crowds out the ones with the highest loss cost, which is backwards. Per-namespace reserves would make the tradeoff explicit, at the cost of a more complicated admission rule.
Whether the cap should scale with the store's configured size rather than being a fixed 2,500.
Scope
This is a
storeconcern, not asummarizeone, and it predates #234 in mechanism — that PR raised the write rate against an existing shared budget rather than introducing the sharing. Item 1 is independent of the rest and could ship on its own.Found by the review on #234, which went looking for a memory leak in the new namespaces, correctly reported that there is none (the TTL still applies), and found this instead.