feat(summarize): the pre-expiry summary gate — compact only when the write was already due, off the hot path

[amiddavid]

summarize now compacts only at a moment when the cache write it costs was going to be paid anyway, and produces the summary off the hot path so that deciding to compact cannot itself become the expensive outcome. A dashboard panel measures what each summary actually earned.

Three defects were found and fixed by this work, two of them in the first version of it. They are the most interesting part of the diff.

The rule

Only ever spend a cache write that was going to be spent anyway.

Compacting rewrites history, and rewriting history the provider is holding in its prompt cache invalidates that entry — the next turn re-writes the whole suffix at 1.25x fresh. There are two moments when that costs nothing:

Cache state	Without compaction	With compaction
warm	a cheap read	a write, and a live entry destroyed	the only harmful case
near expiry	a read now; a full rewrite later if the next turn is late	a small write now, cheap reads after	better, given an assumption about the next turn's timing
already expired	a full rewrite — the entry is gone, this turn pays a write regardless	a small write	strictly better, unconditionally

So the default is cache_state: pre_expiry_or_cold — compact whenever it cannot hurt. Note the ordering: expired is the reliable half and pre-expiry is the half carrying an assumption. The first version of this PR had that backwards, because it reasoned about not destroying something valuable rather than about riding a payment already being made.

Expired is also the reachable half. Pre-expiry needs a request to land inside a 60-second slice of a 5-minute lifetime; any gap past the TTL produces an expired turn on the next request.

Cold is decided by the clock, never by the flag. Ctx.ColdCache is a known false positive on keep-alive'd sessions — proxy/keepalive.go never updates the turn tracker, the −$708 mechanism promexport.go:807 documents — so trusting it would compact live prefixes on exactly the sessions someone is paying pings to protect. coldByArithmetic requires a known TTL and idle time whose difference is ≤ 0.

Defect 1 — the fill fraction could never fire (85130c6)

min_request_frac compared frac × window against schema.MessagesTokens. Two different rulers: MessagesTokens counts message text only, while a context window is stated in the tokens the provider bills. This repo already measures the gap — dash/overview.go's EstimatorDivergence, over requests where nothing was compacted, reports a median 3.38x.

So 0.9 × 1,000,000 really asked for ~3M provider tokens on a 1M model. The default this PR installs could not fire on any model. It went unnoticed because the tests were written in the broken units too: "0.6 of 200k does not fire on a ~4k request" is true under both the wrong arithmetic and the right one.

Fixed by measuring the fill the way the window is measured: Ctx.PrevBilledInput carries the provider's own count for the session's previous turn, written on the response path under the session id apply itself derived. One turn stale by construction and deliberately so — a transcript only grows, so it is a sound lower bound, and a gate that opens one turn late is the harmless direction. Fires now ANDs two separate conjuncts instead of max()-ing them, because taking the larger of two numbers on two different rulers is meaningless; min_request_tokens keeps its own ruler unchanged.

Defect 2 — a slow summarizer turned the best case into the worst (ab545aa)

Found by running it. haiku, real 200k window, 196,208 billed tokens, 250s wait to land in the window:

Turn	What happened
18	trigger fired; the summarizer call hung for its full 300s budget (cg_latency_ms=300,128 against upstream_ms=2,838). Failed open, so the full 197k transcript went upstream — and the entry, which had ~50s of life on arrival, had died during our own stall. 197,879 tokens written, model call wasted.
19	arrived 310s idle → Cold → pre_expiry didn't permit Cold → declined in 3ms. No summary.
20	400 prompt is too long: 203,705 tokens > 200,000 maximum

summarizeCallTimeout is 300s while pre_expiry_seconds is 60, so a slow call guaranteed the entry died while we held the request. The same call shape measured directly completes in 5–14s on haiku (6–19s on sonnet — haiku is the faster choice for it, which also answers whether the model was to blame). The hang itself is unexplained; gateway queueing after a 17-request burst is the leading hypothesis and is unproven.

It doesn't need explaining to be fixed. The firing turn now starts the work on a detached context and forwards immediately; a later turn splices the result, waiting up to summary_wait_seconds (120) if one is still in flight. That deletes the defect rather than tuning it — there is no budget to get wrong on the hot path because the hot path does not wait.

• Single-flight per session, which is not an optimisation: without it a session that keeps firing queues one ~48k-token call per turn and the last writer's checkpoint wins arbitrarily.
• The detached call's budget is summarizeCallTimeout — the same one CONTEXT_GURU_SUMMARIZE_TIMEOUT tunes. A second private constant was my first move and was wrong twice: an operator tuning the summarize timeout expects it to apply, and a budget unreachable from config cannot be shortened when a provider goes bad. "Landing late is still useful" is already expressed by the wait cap being shorter than the budget.
• No keep-alive ping, and the reasoning is in the code so nobody adds one: a ping holds the old, full prefix alive at 0.1x — the prefix we are about to replace. Expiry costs nothing once a summary exists.
• A request with no session id is summarized inline. The checkpoint is keyed by session, so with no key there is no later turn that could ever find the result; async would spend a call and discard it every turn forever. The library API and /compact are unchanged.

Expect one turn per episode to stall a few seconds. The trigger fires only after a 240s+ idle gap, so the user is active immediately after and the next request usually arrives while the call runs. Ten seconds against a 197k write (~$0.22 on haiku, ~$2.70 on Opus at 0.9 of 1M) is a good trade on a turn already waiting on a model.

Defect 3 — declining on a gated turn sent the transcript full

The original control flow returned before the checkpoint replay. Harmless under a size threshold, which is roughly monotone; fatal under a gate true for seconds at a time — turn N sends [head, summary, tail], turn N+1 sends the transcript full, and the provider re-writes the whole suffix. The feature would have paid a cache-write on nearly every turn to save one on a few. Offload now gates without returning: the gates suppress the model call, never the splice.

Measuring what it earned

A "What each summary earned" panel on the Components tab follows every summary for 10% more of the model's context window: cold rewrites avoided, warm reads made cheaper, minus what the summary itself cost.

It introduces no new pricing, which is why it can be trusted — Event.baselineDeltaUSD and repeatRate already price the counterfactual per request at the rates then in force, and cache_miss_reason already labels every turn. The panel scopes existing columns into spans and splits them by the cache state of the turn that earned them.

The accounting rules that took argument:

• The compaction turn is debit-only. At that moment nothing has been saved — only spent. Crediting it front-loaded a saving at the cache-creation rate for work that hadn't paid off.
• prefix_change turns are credited to nothing, and their writes are debited: inside a summarized span the thing that changes the prompt is us. Turn 18 above was recorded as prefix_change, which is exactly the case.
• The invalidation debit comes from cache_write, not the label — a partial hit reads as hit, so a label-derived debit would be zero on the turns where we rewrote a live prefix. Stated as an upper bound.
• Unfinished and voided spans carry a net, reported apart from the settled total. Excluding them was survivorship one level below the coverage line — an unfinished span is exactly where we paid and haven't recouped. Not automatically a loss: we're behind only when 1.25C > 0.1F, so the net is computed rather than presumed.
• Coverage is on the same panel, including the unflattering number: qualifying conversations that produced no summary, and what their expired-cache rewrites cost.

Tests

Every test asserting a summary appears on the triggering turn was migrated, not deleted — the property is unchanged, only the turn it lands on moved. commissionThenSplice is the new primitive, and the drain helpers wait on the production channel rather than sleeping, so a passing test exercises the real synchronisation.

TestSummarizeTimeoutIsCountedAndLeavesInputIntact now pins the opposite contract to the one it used to: a blown deadline must not reach the caller, must leave no checkpoint, and must move the counter — which is the only place a degraded summarizer shows up now that nothing waits for it.

Nine other tests broke when the fill default landed, all anti-vacuity guards firing because their fixtures have no resolvable window. Each now sets min_request_frac: 0. One of them, apply/writeback_test.go, was passing while its premise had silently died.

Verification

gofmt · go vet ./... · go test ./... · go test -race over components/apply/dash (no data races) · static pure-Go build · CGO_ENABLED=0 go test -p 1 across config/components/apply/proxy/store/dash · mkdocs build --strict. All clean.

Design and what is still open

The full design is docs/proposals/pre-expiry-summary-gate.md; the validation plan is docs/proposals/timely-compact-validation.md.

Done, after review: an end-to-end acceptance run against a real Claude Code session on claude-haiku-4-5, with a cold turn after the summary landed — full results. cold_credit_usd = $0.128, hand-checked against the raw rows to the last digit, on a turn that re-created a 32,123-token compacted prefix instead of the ~149,363-token full one. invalidation_debit_usd is $0 there because t0 was itself an expiry — the case that previously reported a loss on the cheapest moment to compact. new_content_billed is 3,486 against a 20,000 target after five turns, so the span axis behaves at production scale where the pre-review version closed on turn 2. And the hot path did not wait: 55 ms, against 300,128 ms on the synchronous run that motivated the change.

One imprecision worth stating rather than glossing: the credit's QUANTITY is summarize's own message-text token count, not the billed difference — turn 17's billed arithmetic gives ~$0.147 against the reported $0.128. Pre-existing behaviour of the savings pipeline (#240), the same two-rulers issue in a third place, and now stated on the panel itself in credit_quantity_note.

Its RATE was separately wrong, and a second review caught it. The panel inherited request_components.saved_usd, whose unique × cacheWriteRate term has no meaning inside an episode — nothing summarize removes there is new content, which is what makes it summarizable. Going async is what exposed it: the stash is written by the detached goroutine, which has no Report, so the checkpoint key first reaches rep.CacheKeys on the turn that replays it, MarkUnique calls the whole removal new, and unlike t0 that turn is credited. Measured on live rows — the same 123,251-token removal on two consecutive cache hits:

row	saved_unique	saved_usd
#44	123,251	$0.15406375
#45	0	$0.01232510

Exactly 12.5x apart on identical content. creditTurn now prices the credit itself, from the removed tokens at the rate that turn's own verdict earns. Two further defects in the same direction went with it: the summarizer's own call was never debited on the async path (its cost lands on t0+1, which the walk classified as a replay), and a turn that closed one span while opening another was debited twice. Together these had a measured episode reporting +$0.376 where the truth was −$0.0397 — wrong sign.

Now measured, and it is the most important number here: the firing rate is effectively zero on continuous agent traffic. A second review built a live arm; it is now checked in as scripts/scenarios/a-firing-rate.sh and documented in docs/proposals/timely-compact-validation.md. Shipped defaults, 53 turns, real Claude Code on claude-haiku-4-5, no injected idle:

peak fill                0.996     <-- the gate needs 0.900
turns at or over the fill gate   12 of 53
cache verdicts           {'hit': 50, 'cold_start': 1, 'unknown': 1, 'prefix_change': 1}
TURNS THAT FIRED         0 of 53
gates (summed)  {'below_request_trigger': 40, 'cache_state_declined_warm': 51, 'window_not_exact': 2}
inter-turn gaps (s)      min=0.1 median=7.3 max=44.3
  gaps >= 240s (the idle pre_expiry needs)   0 of 52
CLIENT COMPACTED         yes — client's own ceiling 199,184 = 0.996 of the window

The fill fraction is not the obstacle. The session passed 0.9, twelve turns qualified, and Claude Code let the context reach 99.6% of the window before compacting — so the theory that the client caps below our gate is refuted on this model and version. The binding constraint is cache_state_declined_warm on 51 turns: pre_expiry needs roughly 240s of idle and the largest gap in a working session was 44s. An active agent keeps hitting its own cache.

So the shipped default is, in practice, gated on a person stepping away for five minutes and coming back to a nearly-full context. That is a real case, and arguably the most valuable one — it is exactly when a cold rewrite of a 199k prefix costs the most — but it is not a general win and this PR should not be read as claiming one. Three options, none of which this branch takes on its own authority: lower the fraction, widen pre_expiry_seconds, or accept the feature as a narrow safety net and document it as one.

A second measured finding, about the panel's span rather than the trigger. scripts/scenarios/b-cold-events.sh forces three cache expiries after a summary. The cold credit accumulates and hand-checks exactly — 3 × 123,251 × 1.25e-06 = $0.46219125, matching the panel to the last digit, each event a prevented rewrite of ~115,000 tokens. But at the shipped span=0.10 the cold bucket is $0.00, because the episode closes after two warm turns (20,095 of a 20,000 target, ~30 seconds of wall time) while a cache expiry needs six minutes. This PR already fixed one reason that bucket was structurally empty (cumulative spend closed the span in one turn); this is a second, and it is about the 10% figure being short relative to the timescale a cache entry lives on rather than about the arithmetic.

Both are arguments about the shipped defaults, measured rather than asserted, and both are the repo owner's call.

Filed separately, deliberately not on this branch: #233 (DefaultStatic() reports 200K for every 1M Claude model), #235 (two install.sh tests fail on any host with the binary already on PATH), #238 (extend an existing summary instead of discarding it when the covered span moves), #239 (learn the client's own compaction point instead of assuming a fraction of the model window), #240 (saved_usd prices our token count at the provider's rate), #241 (offer provider-native compaction where no summarizer model exists), #243 (the cold gate trusts a clock keepalive.go never updates — a design decision, not a patch), #244 (the store's pin budget saturates silently and no metric says so), #245 (the cache gate is wrong on OpenAI-shaped backends in both modes, in opposite directions), #246 (seven components/offload tests fail on main at -count=2), #247 (split Trigger so extract/extract_llm cannot accept cache_state keys they ignore).

🤖 Generated with Claude Code

[amiddavid]

The 0.9 default could not fire. Fixed in 85130c6.

Found while building the dashboard measurement for this trigger: min_request_frac compared frac × window against schema.MessagesTokens. Those are two different rulers, and the mismatch made the fraction unreachable, not merely inaccurate.

MessagesTokens counts message text only — no system prompt, no tool declarations, no JSON envelope. A context window is stated in the tokens the provider bills, which include all of it. This repo already measures the gap: dash/overview.go's EstimatorDivergence, computed over requests where nothing was compacted so the two describe the same prompt, reports a median 3.38x (p25 2.43, p90 6.80).

So MessagesTokens >= 0.9 × 1,000,000 really asked for ~3M provider tokens on a 1M model — the request is rejected upstream, or the client compacts, long before it can be true. On a 200k model 0.9 lands at ~608k billed against a 200k limit. The default this PR installs could not fire on any model.

It went unnoticed because the tests were written in the broken units too: they asserted "0.6 of 200k does not fire on a ~4k request", which is true under both the wrong arithmetic and the right one.

The fix

Measure the fill the way the window is measured — the provider's own reported input count:

• Ctx.PrevBilledInput carries fresh + cache-read + cache-write for the session's previous turn. One turn stale by construction and deliberately so: a transcript only grows, so it is a sound lower bound, and a gate that opens one turn late is the harmless direction.
• apply.RecordBilledInput is the only writer, exported so the host calls it from the response path under the session id apply itself derived (Trace.Session). Re-deriving the key from the outgoing body would hash a transcript this pipeline had just rewritten — a different key on exactly the sessions where compaction happens, and a permanent zero that reads as "not full".
• store.BilledPrefix is pinned, for the same reason SumPrefix is: the trigger declines when the figure is absent, so losing the key turns the component off rather than costing a re-measurement.
• Trigger.Fires takes the Ctx and ANDs two separate conjuncts instead of max()-ing them — taking the larger of two numbers on two different rulers is meaningless. min_request_tokens keeps its own ruler unchanged, so no existing config that sets it changes meaning.
• FracResolvable also requires PrevBilledInput > 0, so a session's first turn counts window_not_exact rather than reading an unknown fill as an empty one.

Tests

Rewritten in the units that matter, with the case the old ones could not express: a request whose message text is nowhere near the threshold, on a session the provider has already billed past it, must fire. The fill-conjunct table now asserts as a precondition that the fixture's MessagesTokens sits below the floor, so it cannot quietly stop being about the ruler.

Verified on a clean PATH: gofmt -l clean, go vet ./... clean, go test -race ./... clean tree-wide, static pure-Go build, CGO_ENABLED=0 go test -p 1 over config/components/apply/proxy/store clean, mkdocs build --strict clean.

Two install.sh tests in context-guru-plugin fail on any host that already has context-guru-proxy on PATH — unrelated to this diff, filed as #235.

Still not measured

The firing rate remains unmeasured, and this defect is why it matters: the mechanism is only now capable of firing. The condition has to be met once per session rather than per turn, so the population that gets no benefit is sessions that reach the fill threshold and never have a turn land near cache expiry.

🤖 Generated with Claude Code

[amiddavid]

Review round 1 of 3 — three findings, one of which repeats Defect 1's shape

I read the async path (summarize.go Offload, summarize_async.go), the gate (trigger.go, cachephase.go), the wiring (apply.RecordBilledInput, proxy.resolveWindow, proxy.serve) and the measurement (dash/compactepisode.go) against its tests. The mechanism is sound: single-flight, the detached sink, causedWriteUSD's cold-t0 rule, and "gate without returning" are all right, and I could not find a route where a lossy splice escapes without a stash. Three things below, ranked.

---

1. The episode span closes on the FIRST turn after t0, so the panel measures ~one turn — not "several cold misses"

dash/compactepisode.go:257 documents defaultSpanFrac = 0.10 as "long enough to contain several cold misses on real traffic", and the span now closes on cumulative billed input (cur.SpentBilled += r.Billed, SpentBilled >= span), where billed is fresh + cache_read + cache_write (compactEpisodeSelect, :704).

Those two cannot both hold. Billed input re-counts the whole prefix every turn, so a single turn of a qualifying conversation is already several times 0.1 × window. On the very run that motivated this file — haiku, 200,000 window, post-summary turns of 31,682 — the span is 20,000 and closes on the turn immediately after t0. A cold miss needs a 5-minute idle gap; a two-turn span measures seconds of wall clock, so ColdCreditUSD is structurally near-empty and the headline bucket reports approximately nothing.

The move from per-turn size to cumulative spend was right (a per-turn axis never closes — TestTheSpanClosesOnCumulativeSpendEvenWhenEachTurnGetsSmaller earns its place). What did not follow is the threshold: the axis changed from transcript growth to spend, and spanFrac kept its growth-shaped value. queryFrac (:797) rejects f >= 1, so an operator cannot even work around it by asking for a span of 3× the window.

Why the tests do not show it: every span fixture keeps t0 tiny and the window at 1,000,000 while using the live run's per-turn sizes. TestAnEpisodeClosesWhen...'s own comment says it outright — "t0 is deliberately SMALL: cumulative spend closes the span, so a large t0 would close it on its own turn and leave no span to measure." That is the fixture reporting the defect. At the live run's real window (200,000, not 1M) the same rows close on turn 2. This is Defect 1's shape one level up: a threshold stated in one quantity, evaluated against another, with fixtures whose units make the mismatch invisible.

Two directions, either fine by me: keep spend as the axis and state the threshold as a multiple of the window (widen queryFrac, and rewrite the "several cold misses" rationale into whatever the new number actually buys); or keep "10% more of the window" and close on a growth proxy that is still monotone — cumulative fresh_input + cache_write approximates new content and excludes the re-read prefix. What must not ship is the current pair plus a comment claiming a span it does not have. A test at production scale (real window, real t0 billed, several turns) would have caught it and should pin whichever you pick.

2. observe mode can never fire the new default, so the projection an operator decides on reports zero

PrevBilledInput has exactly one writer: proxy.go:1407, on the enforced response path, with tr.Session, into tn.Store. In observe mode applyMode returns apply.Trace{} (modes.go:41-42) — so tr.Session is "" and the write no-ops — and the off-path run reads from tn.Shadow (modes.go:145), a different store, which nothing ever writes cg:bin: into.

So on an observe-mode tenant PrevBilledInput is permanently 0 → FracResolvable false → window_not_exact on every turn → with the shipped min_request_frac: 0.9 default, summarize projects zero saving, forever. That is the same defect this PR fixes for /compact (resolveWindow's own comment: "silently disabled every fraction-based trigger on the endpoint the offline eval runs against, so eval measured a different component than the one that ships"), arriving through the other door — and observe mode is precisely where someone decides whether to turn this on.

The fix wants the shadow run to record its own billed input under the shadow store and the session id that run derived (the observe path has the usage; it just discards the trace). Whatever the mechanism, the gate should not silently read "not full" in a mode whose whole purpose is measuring what would have happened.

3. Detached summarizer calls have no global bound and no drain

Single-flight is per session, which is the right unit for correctness, but nothing caps concurrency across sessions. The inline path was self-limiting: a call occupied a request, and its cost showed up in that request's row and latency. Now a burst of sessions returning from idle can each commission a ~57k-token call at once, with no ceiling, no visible latency, and — until each session's next turn — no row anywhere. asyncStarted/asyncCommitted show the gap after the fact; they cannot refuse.

Related, same root: nothing drains on shutdown. A commissioned call in flight when the process exits is money spent, no checkpoint, and no record — the pair of counters is process-local, so it does not even show up as a gap. A global semaphore (rejecting to rep.Gate rather than queueing, so a saturated proxy just declines to compact) plus either a drain or a counter for "commissioned, never resolved" would close both.

---

Smaller, non-blocking

• takeDeferredUsage can drop a summarizer cost it was written to preserve (summarize_async.go). It is Get → Put("{}") → replay, while deferUsage on a background goroutine does its own Get → add → Put for the same key. A call finishing between our Get and our Put has its record erased unread — the cost silently vanishes, which is the one direction the panel must not lean, and it is the exact failure deferUsage's "additive rather than replacing" comment guards against in the other direction. A read-and-clear the store performs atomically, or a per-session mutex around both, removes it.
• extract_llm shares Fires' new semantics without adopting FracResolvable (extract_llm.go:655). min_request_frac there is now compared against PrevBilledInput, and Fires skips the conjunct when that is 0 — so on an extract config that sets the fraction, an unknown fill permits where summarize declines. docs/components/extract_llm.md documents the AND but not that asymmetry. Either have extract consult FracResolvable too, or say in the doc that its fraction is best-effort.
• waitFor's parameter is named cap, shadowing the builtin inside a function that does no appending. Harmless, but the file is otherwise fastidious about names (st vs store).

Nothing here touches the fail-open property or the reversibility invariant, and #1/#2 are both measurement rather than request-path defects — but #1 is the panel this PR is partly justified by, and #2 is how someone else decides to adopt it.

[amiddavid]

Round 1 addressed in 372fa22

All three ranked findings and all three smaller notes. Two of the three were mine and both diagnoses were exactly right, including one where my own fixture comment stated the defect out loud.

1. Span axis — fixed, and neither obvious axis worked

You're right, and TestAnEpisodeClosesWhen...'s comment was the confession: "t0 is deliberately SMALL: cumulative spend closes the span, so a large t0 would close it on its own turn." At the live run's real 200,000 window the span was 20,000 against a next-turn 31,682 — closed on the turn after t0, seconds of wall clock, so ColdCreditUSD was structurally empty. Defect 1's shape one level up, as you said.

I took your second direction, and it needed a third option rather than either of the two I'd tried:

• per-turn size falls as soon as a summary lands (154,584 → 31,682), so a growth-shaped span never closes;
• cumulative spend re-counts the whole prefix every turn, so one turn exceeds the span.

The axis is now cumulative new content in billed units: fresh_input, plus cache_write only on turns that hit (the newly-written tail). A write on a miss is re-creation of a prefix that already existed and is excluded — counting it calls the whole transcript new every time an entry expires, which is precisely what let a cold t0 close its own span. cache_read is never new.

Your fresh_input + cache_write suggestion is what I started from; the miss/hit split is the one refinement it needed, and t0 on the live run is why — its 167,263-token write is re-creation, not growth.

On the live run's real figures the span now covers tens of turns rather than one: post-summary turns add 3,323 / 158 / 275 of written tail against a 20,000 target.

queryFrac's bound moves 1 → 8 so a span of several windows is askable, as you noted it should be.

TestAtProductionScaleTheSpanSurvivesMoreThanOneTurn is the test that was missing: real window, real t0 billed, real per-turn read/write, asserting the span is still open after four turns and that t0's re-creation contributes zero new content. It fails under the old axis.

2. observe mode — fixed

Correct, and it is the same defect resolveWindow fixes for /compact through the other door, which makes it worse rather than better. apply.SessionIDFor exposes the derivation, BodyOpts now calls the same function so the two cannot drift, and the response path records into tn.Shadow under that id. Derived on the response path, so observe keeps paying only the enqueue — the property modes.go documents and which I'd otherwise have quietly broken to fix this.

3. Bound and drain — bounded; "drain" is a counter, not a drain

maxConcurrentSummaries = 8, acquired non-blocking, rejecting to a gate exactly as you suggested — a proxy saturated enough to fill it should decline to compact rather than queue work nobody waits for. The refusal releases the per-session flight first, or the session wedges reporting the wrong reason forever (asserted).

Two gate names now, because they have different remedies: summary_already_in_flight (ordinary, self-correcting) vs summary_concurrency_full (the deployment is shedding compaction).

On shutdown I added summarizeUnresolved — incremented at commissioning, decremented on resolution — so outstanding calls are visible while the process lives. I did not implement a real drain, and I'd rather say so than imply otherwise: a call in flight at exit is still money spent with no checkpoint, and the counter dies with the process just as you observed. If you think that's insufficient for round 2, the honest fix is a shutdown hook that waits briefly on summarySlots, and I'll add it.

Smaller — all three

• takeDeferredUsage race: real, and it loses a cost, which is the direction the panel must not lean. deferredUsageMu serializes both halves; the lock releases as soon as the record is cleared, since the replay touches only that request's own sink.
• extract_llm asymmetry: left best-effort, and now says so in the code and in its doc table. Adopting FracResolvable there would make it fire less on exactly the deployments that cannot report a billed figure, for no correctness gain — its real gate is the per-candidate economics. Documented rather than silently divergent.
• cap → limit.

---

Verified: gofmt, go vet, go test ./..., go test -race over components/apply/dash, mkdocs --strict. All clean.

Two things I have not resolved and am not claiming as done: ColdCreditUSD is still unexercised on live data (every post-summary turn in the validation run was warm, so only unit tests cover the headline bucket), and the firing rate remains unmeasured. Both are in the PR description as open.

🤖 Generated with Claude Code

[amiddavid]

Review round 2 of 3 — the span axis is right; the counters it all rests on reach nobody

Verified in 372fa22:

• Span axis (feat: Go context-engineering proxy/library (tree-sitter, TOON, Starlark extractor, config, metrics, real integrations) #1) — the new axis is correct and the refinement you added is the right one: newContentBilled counts a write only on a turn that hit, so t0's 167,263-token re-creation cannot close its own span. TestAtProductionScaleTheSpanSurvivesMoreThanOneTurn genuinely pins it — under the old axis turn 2's 32,056 clears the 20,000 span and the episode closes, so the EpisodeOpen assertion fails. queryFrac at 8, SpanMeasureNote and the defaultSpanFrac rationale all now say what the code does, and the "several cold misses" claim is replaced with the honest "not yet shown on real traffic".
• observe (feat(engine): unify compaction under one Compactor abstraction; add summarizer + truncation; comprehensive stats #2) — the shadow record lands under the id the shadow pipeline derives, and deriving it on the response path keeps observe paying only the enqueue. I checked the drift risk directly: msgsRaw is parsed before splitVolatileTail, and the split touches system/metadata rather than messages, so the two derivations do agree today.
• Bound (Feat: runtime reduction mode API (on/off/deterministic) #3) — non-blocking acquire, refusal counted, and the per-session flight released before returning so the session cannot wedge; the test asserts that release specifically. Defer order is right: recover is registered last so it runs first, and the slot is released even on a panic. Two gate names is the correct call.
• Usage race — the mutex closes it, and unlocking at the clear rather than after the replay is the right scope.

Four things below. The first is the one that matters, and it undoes most of what #3's fix was for.

---

1. AsyncSummaryStats has no caller — every async counter is unreachable

$ grep -rn "AsyncSummaryStats" --include='*.go' . | grep -v summarize_async.go
$

Nothing reads it. summarizeAsyncStarted, asyncCommitted, waitedMs, waitTimeouts, asyncRefused and summarizeUnresolved are computed, atomically maintained, and served to no one. The existing counters do have a route — proxy.go:1959 puts SummarizeTimeouts/Errors/CallTimeoutMs into metrics.Snapshot (metrics.go:699-701) — and the async ones were never added beside them, while three separate comments say they were: "exported through /stats beside the existing summarize ones", "these two counters are the ONLY place a degraded summarizer shows up", "/stats reports a non-zero value only while calls are genuinely outstanding".

This is not cosmetic, because it is the load-bearing claim of the whole async design. Nothing on the hot path waits for a summary any more, so the started/committed gap is by construction the only way anyone learns that calls are being commissioned and lost — and it goes nowhere. Same for asyncRefused, which is how an operator would learn the deployment is shedding compaction, and for summarizeUnresolved, whose entire purpose is being watched. The signature change from 4 to 6 returns compiling untouched is itself the evidence: there was no caller to break.

Wire the six into metrics.Snapshot next to the three that work, and pin the route with a test — this is precisely the vacuity that TestSummarizeTimeoutIsCountedAndLeavesInputIntact exists to prevent for the counter that does ship.

2. summarizeUnresolved cannot show what a restart lost, and its comment says it can

"This counter is written at the moment of commissioning and cleared on resolution, so /stats reports a non-zero value only while calls are genuinely outstanding — an operator watching it across a restart sees what was lost."

The first half is true. The second cannot be: the counter is process memory, so after the restart it reads 0 and the calls that died are exactly the ones it can no longer describe. Combined with #1 it currently describes nothing at all.

On your drain question — I would not add the shutdown hook. A log line at commission and at resolution is the better answer: it survives the restart the counter cannot, it makes the unmatched pair queryable after the fact (which is what "an operator sees what was lost" actually requires), and it does not add a shutdown path that can hang while holding the process open for a cost-saving measure nobody is waiting for. The loss being bounded — one call, rare, seconds-wide — is what makes the cheap answer the right one. Either way, that clause of the comment has to go or become true.

3. SessionIDFor's central claim is false — the derivation is duplicated

"The derivation is not duplicated — BodyOpts calls this same function — because two copies of a session id are two ids the moment either changes."

BodyOpts still computes session.Scoped(o.Tenant, explicitSession(o.Session, body), sys, firstUser) inline at apply.go:470. The two copies already differ: SessionIDFor falls back to input when messages is absent (:1121-1123), while BodyOpts requires an array messages and returns early otherwise (:415-416). So a body shaped that way gets a billed figure recorded under an id no pipeline will ever read — the same silent zero, one door further along.

Harmless today for Anthropic-shaped traffic, but the comment states an invariant the code does not hold, and that is the version a future reader will trust. Have BodyOpts call it (or factor the shared tail so both take the same norm), so the property is structural like the comment says.

4. The observe fix has no test

Round 1's #2 was a defect whose whole signature was that it looked like it was working, and the fix ships with nothing pinning it. One test — after a response on an observe tenant, prevBilledInput under the shadow store is non-zero under the id the shadow run derives, or equivalently a second observe turn does not gate window_not_exact — is what stops it silently reverting to a permanent zero. proxy/cgllm_test.go already drives responses through this path.

---

Minor

newContentBilled assumes content is written on arrival, which is what your live data shows (fresh is 3 tokens on each warm turn, so the tail is billed as cache_write once and read thereafter). Where a deployment's breakpoints lag the tail, the same tokens are billed fresh on arrival and written when a breakpoint later covers them, and this counts both — halving the effective span. Not worth a code change, but the function asserts its classification quite absolutely for something that holds in one breakpoint regime; a sentence naming the assumption would keep the next reader from treating it as exact.

---

I have one round left. Nothing here is a request-path correctness risk: #1 and #2 are visibility, #3 is an invariant that is claimed rather than held, #4 is regression cover. Wiring the counters is the one I would not ship without — the async design's answer to "how would you know it broke" is currently "you would not".

[amiddavid]

Round 2 addressed in cadf559

All four, plus the minor. Thank you for verifying round 1's fixes rather than taking them on trust — particularly checking that the production-scale test fails under the old axis, which is the only thing that makes it worth having.

1. The counters reached nobody — fixed, and the proof you gave is the damning part

AsyncSummaryStats had no caller anywhere, and the 4→6 signature change compiling untouched is exactly the evidence. Worse than an ordinary dead accessor, for the reason you named: producing the summary off the hot path removed every other way to see that path working, so these counters aren't decoration — they're the only signal, and they pointed nowhere.

Seven fields on /stats now: the six counts plus the concurrency bound, which travels beside them for the reason llm_call_timeout_ms travels with its timeout total. TestAsyncSummaryCountersReachStats pins the route as a table, so a seventh counter added without a snapshot field fails there rather than going nowhere again.

Two repo guards fired and both were right — promexport_coverage_test and stats_golden_test. On Prometheus I followed the notExportedWhy block's own preamble rather than my instinct: seven new cg_* series do not belong in a PR about the trigger. They're listed with reasons, and I named Started-vs-Committed as the pair to export first, since a growing gap between them is the only signal that a detached summary was paid for and lost.

2. Taking your recommendation over my own offer

You're right that the comment claimed something a process-memory counter cannot do, and right that the shutdown hook is the wrong fix — a path that can hang the process for a cost-saving measure nobody is awaiting. No hook. Commissioning and resolution each emit a log line instead, so an unmatched pair survives the restart and is queryable afterwards. The resolution line is in a defer so it fires on every exit — success, model error, timeout, recovered panic — because a missed line would fake the very signal being preserved.

3. Structural, not re-commented

Correct on both counts, and the second half is the sharper one: the copies had already drifted, on which body shape they accept, so a body carrying input recorded under an id nothing reads. Both individually reasonable, which is why it was invisible.

Two shared pieces now, and BodyOpts and SessionIDFor both call both:

• messagesArray — the one gate on a body's message array.
• sessionIDFrom — the one derivation of the id from normalized messages.

There is no copy left to drift. I'd rather have this than a corrected comment, which is what you were pointing at.

4. The observe test exists, and I checked it isn't vacuous

TestObserveModeRecordsBilledInputIntoTheShadowStore drives a real observe-mode request through New (so it reads the shadow store the request path actually writes, not an injected one) and asserts the cg:bin: record lands under the id the observe run derives.

Verified to fail with the fix disabled and pass with it enabled — a permanent-zero defect deserves a test that demonstrably catches it coming back.

Minor — named, with its direction

newContentBilled does assume write-on-arrival. Named in the comment, along with which way it errs: where breakpoints lag, the same tokens are billed fresh then written and count twice, closing the span in roughly half the new content it claims — a shorter span, so it under-reports this component rather than over-reporting it. A client that never writes at all degenerates to counting fresh input only, which is correct.

---

Verified: gofmt, go vet, go test ./..., go test -race over components/apply/dash. All clean.

Still open and unchanged, so round 3 doesn't have to rediscover them: ColdCreditUSD has no live-data coverage (every post-summary turn in the validation run was warm), and the firing rate is unmeasured. Both are in the PR description. The learned-compaction-point work is filed as #239 rather than grown into this PR, and the static-table defect remains #233.

🤖 Generated with Claude Code

[amiddavid]

Review round 3 of 3 — signing off

Verified in cadf559:

• Counters (feat: Go context-engineering proxy/library (tree-sitter, TOON, Starlark extractor, config, metrics, real integrations) #1) — the seven fields are wired in the real handler, and I checked the multi-assign ordering against AsyncSummaryStats' return order: started, committed, waitedMs, waitTimeouts, refused, unresolved lands on Started, Committed, AwaitedMs, AwaitTimeouts, Refused, Unresolved. Correct. stats_golden_test.go hits the actual /stats handler and asserts all seven keys, so the route from Snapshot to the wire is now pinned by the real path rather than by a claim. Deferring the Prometheus series to a later PR under this block's own preamble, with Started-vs-Committed named as the pair to export first, is the right call inside a trigger PR — the review asked for the numbers to be reachable, not for seven new cg_* series.
• Unresolved (feat(engine): unify compaction under one Compactor abstraction; add summarizer + truncation; comprehensive stats #2) — the comment now says what the counter can and cannot do, the log pair is the durable record, and the resolution line is in the defer that runs after recover, so a panicking goroutine still emits it. I checked the defer order: recover is registered last and runs first, then the counter/slot/log defer, then finish. The no-drain decision is now documented as deliberate with its reason, which is what I wanted rather than the hook.
• Session id (Feat: runtime reduction mode API (on/off/deterministic) #3) — messagesArray and sessionIDFrom are genuinely single, both callers reach the id through them, and the input fallback is gone rather than papered over. The comment describes the code now.
• observe (feat: redesign as context-guru — provider-agnostic context-engineering #4) — the test reads h.shadow, the store the request path actually writes, rather than an injected double, and you confirmed it fails with the fix disabled. That is the coverage the defect needed.
• Minor — newContentBilled names the breakpoint assumption and which way it errs. Stating that it under-reports rather than over-reports is the part that matters.

One small note, not worth a fourth round: TestAsyncSummaryCountersReachStats re-executes the same assignment expression the handler contains rather than calling the handler, so it pins a copy of the route. A field swapped in proxy.go alone (Refused reading waitTimeouts, say) would pass it. The golden test already guarantees the keys exist on real output, so the residual gap is only the field↔source pairing — closable whenever someone is next in the file by asserting summarize_async_concurrency == 8 off decoded /stats JSON, which is the one value that is a known constant.

---

Sign-off

Across three rounds this PR fixed: a measurement axis that closed every span on the first turn after t0 (with a production-scale test that fails under the old axis), a permanent zero in the mode operators use to decide adoption, an unbounded detached-call path, a lost-update race on the summarizer's own cost, six counters that reached nobody, and a duplicated session-id derivation that had already drifted. None of those were request-path correctness defects, and the fail-open and reversibility invariants held throughout every version I read — but four of the six were silent-zero failures, which is the class this component can least afford, and all four are now pinned by a test that discriminates.

Approving. Two things are open by explicit acknowledgement rather than oversight, both in the PR description, and neither blocks: ColdCreditUSD has no live-data coverage (every post-summary turn in the validation run was warm), and the firing rate is unmeasured. The headline bucket of the panel is therefore still unexercised on real traffic — the end-to-end Claude Code run the description names as next is what should confirm it, and I would treat a first run that reports a non-empty ColdCreditUSD as the real acceptance test of the span axis this review changed twice.

[amiddavid]

Round 3's note closed in 3c2e226

You left it as "your call" — it was worth doing now, because the criticism identifies the test as an instance of the very defect it was written to catch.

TestAsyncSummaryCountersReachStats re-executed the handler's own assignment expression against a local Snapshot and compared the result to AsyncSummaryStats. So it pinned a copy of the route: swapping two fields in proxy.go alone would have passed, because the test performed the same swap. A check that cannot fail for the thing it names is worse than a weak one, and the entire reason these counters needed pinning is that they were maintained and read by nobody while three comments claimed otherwise.

It now calls h.stats and decodes the JSON, as stats_golden_test does:

• every async field present on the wire, not merely in the struct;
• summarize_async_concurrency equals offload.MaxConcurrentSummaries().

The bound is what makes the pairing assertable at all — a distinctive constant, so reading it back proves the field carries its source rather than merely existing. The six counts are zero in a fresh process, so key existence is all they can honestly be held to, and the golden test already pins that over the real handler.

Verified against your exact scenario: mis-wiring SummarizeAsyncConcurrency in proxy.go alone now fails with "on the wire = 0, want 8". The previous version passed.

---

Thanks for three rounds that each found something real. The three that changed my mind rather than my code: the span axis (twice — cumulative spend was my own over-correction, and neither obvious axis worked); observe mode, which was the same defect I'd just fixed for /compact arriving through a door I hadn't checked; and the counters, where your signature-compiles-untouched observation was the proof I couldn't argue with.

Noted for whoever merges

The sign-off is a comment rather than a formal approval because GitHub refuses self-approval — the PR and the reviewing session authenticate as the same account. If a formal approval is required to merge, it has to come from a different account. Recording that here so it isn't mistaken for a missing review.

Open by acknowledgement, not oversight

• ColdCreditUSD has no live-data coverage. Every post-summary turn in the validation run was warm, so the headline bucket is covered only by unit tests. Your framing is the right one: a Claude Code run reporting a non-empty ColdCreditUSD is the real acceptance test of the span axis this review changed twice. That needs a run whose post-summary turns include a genuine idle gap.
• The firing rate is unmeasured. Computable from dash/kvcache.go's per-request IdleMs with nothing shipped.

Both are in the PR description. #239 holds the learned-compaction-point design (which would supersede the fixed 0.9 fraction), #233 the static-table windows, #235 the install.sh test hermeticity, and #238 additive summaries.

🤖 Generated with Claude Code

[amiddavid]

Confirmed in 3c2e226 — the test now calls h.stats and reads the wire, and asserting summarize_async_concurrency against MaxConcurrentSummaries() is the right lever: it is the one field whose value is distinctive, so it proves the pairing where six counters that are all zero in a fresh process cannot. Verifying that the mis-wiring passes before and fails after is what makes the claim checkable rather than asserted.

Agreed that this was worth closing rather than deferring, and for the reason you give: a test that re-performs the swap it is meant to detect is an instance of the defect it was written to catch, which makes it worse than no test — it reads as coverage.

Nothing further from me. My three rounds are used and everything raised in them is addressed; the sign-off stands as posted, with the two acknowledged gaps (ColdCreditUSD unexercised on real traffic, firing rate unmeasured) open by agreement and the end-to-end run — one whose post-summary turns contain a genuine idle gap — as the acceptance test of the span axis.

A formal GitHub approval still has to come from an account other than the author's; that is the repo owner's call.

[amiddavid]

Acceptance run: ColdCreditUSD is non-empty, and hand-checks exactly

The gap both of us recorded as open is now closed. A real Claude Code session on claude-haiku-4-5 (real 200,000 window) through an isolated test proxy, built from 3c2e226, with two gaps past the TTL — the second one after the summary landed, which is what the previous validation run lacked.

19 requests, $0.56.

The turn that earns it

 15  billed=149366  read=0      write=149363  ttl_expiry  ours=105155  cg_ms=55  summary_started   gap=394s
 16  billed=31994   read=28730  write=3261    hit         ours=105250  cg_ms=34  reused_checkpoint gap=9s
 17  billed=32126   read=0      write=32123   ttl_expiry  ours=105367  cg_ms=31  reused_checkpoint gap=397s
 18  billed=32253   read=32123  write=127     hit         ours=105469  cg_ms=31  reused_checkpoint gap=3s
 19  billed=32339   read=32250  write=86      hit         ours=105536  cg_ms=33  reused_checkpoint gap=2s

Turn 17 is the case the headline bucket was built for: the prompt cache expired after the transcript was summarized, so that turn re-created a 32,123-token compacted prefix instead of the ~149,363-token full one. Its saving is what lands in ColdCreditUSD.

Note turn 17's gate is below_request_trigger — the incoming figure is now the compacted 32k, below the fill — and it replayed anyway. That is the gate-without-returning fix doing its job on live traffic, on precisely the turn where reverting to the full transcript would have cost the most.

The panel, at the shipped span

provenance               recorded
state                    open
window                   200000
new_content_billed       3486        (span target 20,000)
turns                    5
cold_credit_usd          0.12834875
read_credit_usd          0.14888455
invalidation_debit_usd   0
net_usd                  0.2772333

Hand-checked against the raw rows rather than read off the panel:

hit          $0.14888455  over 3 turns
ttl_expiry   $0.12834875  over 1 turn

Both buckets agree to the last digit. invalidation_debit_usd is $0 because t0 was itself a ttl_expiry — causedWriteUSD's rule, and the case that previously reported a loss on the cheapest moment to compact.

The span axis, exercised at production scale

new_content_billed is 3,486 against a 20,000 target after five turns — so the episode is correctly still open, where the pre-review axis would have closed it on turn 16. Queried at span=0.002 the same rows produce a closed episode over 2 turns, which is what confirms the closing logic works rather than merely never firing.

Async counters, off the wire

summarize_async_started      1
summarize_async_committed    1
summarize_async_unresolved   0
summarize_async_refused      0
summarize_await_timeouts     0
summarize_async_concurrency  8

Started == committed with nothing unresolved: the detached call was commissioned and landed. cg_ms on the firing turn is 55 ms — the hot path did not wait, where the synchronous version measured 300,128 ms on the run that motivated the change.

One honest imprecision

The credit is priced from summarize's own token count at the provider's rate, not from the billed difference. For turn 17 the billed arithmetic is (149,363 − 32,123) × 1.25/MTok ≈ $0.147 against the reported $0.128 — same order, not the same number, because baselineDeltaUSD counts message-text tokens. That is pre-existing behaviour of the savings pipeline rather than anything this PR introduced, and it is the same two-rulers issue in a third place. Flagging it rather than presenting $0.128 as the billed delta.

What is still open

The firing rate remains unmeasured — this run forced the conditions rather than observing how often they co-occur. Computable from dash/kvcache.go's per-request IdleMs with nothing shipped, and #239 would supersede the fixed fraction it depends on.

🤖 Generated with Claude Code

[amiddavid]

Acceptance confirmed — with one precise residual worth naming

I re-derived the headline rather than reading it: turn 15 wrote 149,363 tokens, turn 17 wrote 32,123, and the 117,240-token difference at haiku's 1.25/MTok write rate is $0.14655 — matching your billed figure. net_usd reconciles too ($0.12834875 + $0.14888455 − 0 = $0.2772333). Turn 17 is the case the bucket was built for, and invalidation_debit_usd = 0 on a ttl_expiry t0 is causedWriteUSD's rule firing on the exact case that previously reported a loss at the cheapest moment to compact.

The detail I would keep in view: the reported credit is 13% below the billed delta, i.e. the two-rulers imprecision under-reports this component. That is the safe direction and it is why the acceptance still stands — but it also means anyone who later "fixes" baselineDeltaUSD moves every savings figure in this repo up, which is the direction that needs the most care to justify. That is now the third site of one root cause (Trigger.Fires, the episode span, and saved_usd), it predates this PR, and it affects every saved_usd consumer rather than this panel — so it wants its own issue with these numbers in it, not a paragraph in a PR comment that scrolls away. You have the measured pair; worth filing while it is in hand.

One residual, small and precise. This run confirms the credit computation on live traffic and the closing logic only at span=0.002. At the shipped span the episode is correctly open, so its money is in OpenNetUSD and the settled NetUSD for this run is $0 — by design, not a gap. So what remains unobserved on real traffic is a closed episode at the shipped span: the thing that feeds the panel's headline total. The axis is verified either way, since closing is span-parametric and you exercised it, so this is a note for whoever next reads the panel expecting a settled figure — not a request.

started == committed, nothing unresolved, and cg_ms of 55 against the 300,128 that motivated the redesign: the async path is doing on live traffic what the design argued it would.

That closes everything I raised. Three rounds, six findings, all addressed with tests that discriminate — and the two that changed the design rather than the code (the span axis and observe mode) were both cases where the first fix was reasoned from the wrong end. Good work. The firing rate stays open by agreement, and the formal approval remains the repo owner's to resolve, since GitHub refuses self-approval on a shared account.

[amiddavid]

Both post-acceptance points actioned

1. The imprecision is now #240

Filed with the measured pair, and you were right that a paragraph here would scroll away — it is the kind of number that is very hard to reconstruct later.

I re-derived it rather than copying your figure, and it comes to 12.42%:

Tokens removed, as the provider billed them	117,240 (149,363 − 32,123)
At haiku's 5m write rate	$0.14655
What saved_usd reported	$0.12834875
Implied token count behind that	102,679
Under-report	12.42%

Your framing of why it matters is the part I made the centre of the issue: it under-reports, so a fix moves every savings figure in the product upward — the direction needing the most justification. Doing that deliberately, with the arithmetic on record, is a very different act from someone later "correcting a tokenizer" and silently inflating the headline.

And your point that it is the third site of one root cause is in the issue as the argument for filing rather than patching: Trigger.Fires (an unreachable gate), the episode span (an empty measurement), and now saved_usd (a quiet 12% understatement). The first two announced themselves by producing nothing; this one survived precisely because it produces something plausible.

Own issue, own branch off main — it predates this PR and touches consumers this PR does not.

2. The residual is now in the operator-facing doc — 64d488f

You were careful to call it a note rather than a request, and it deserved a durable home rather than a PR comment. docs/components/summarize.md now says it directly: on a freshly summarized session the panel shows an open episode with populated credits beside a settled total of $0, and that is correct rather than broken.

The reason is structural and worth the operator knowing: a summarized session adds new content slowly by construction — five turns after the summary on this run had accrued 3,486 of a 20,000 target — so open is the normal early state. The doc also points at ?span= for anyone who wants to see the closing behaviour without waiting, since the axis is the same and only the threshold moves.

I have not claimed a closed episode at the shipped span on real traffic, because there isn't one. The axis is verified in both directions (never-closes and closes-immediately were both real bugs here), but that specific observation stays outstanding and is now written down where someone will hit it.

---

Everything raised across three rounds plus the acceptance run is closed. Two things remain open by agreement and are recorded in the PR description: the firing rate is unmeasured, and the formal approval is blocked by the shared account and is the repo owner's call.

Thanks for re-deriving the headline instead of reading it. The 12.4% gap only surfaced because you did, and it is now the fourth defect this review produced that I would not have found.

🤖 Generated with Claude Code

[OsherElhadad]

Review — four blockers, and a live run that says the mechanism does not currently fire on real traffic

Reviewed by four independent passes over pr234 @ 64d488f vs origin/main @ e66f255: gating/trigger logic, async concurrency, dashboard arithmetic, and an empirical arm driving real claude -p sessions through a locally-built proxy against the production gateway with production's own MODEL_PRICES. Findings are marked CONFIRMED (a test or measurement was run) or SUSPECTED (code reading only).

Credit where it is due, since most of this comment is critical. The hot-path claim is real and measured: the firing turn adds 0.378 ms. recover() is present and correctly ordered in the detached goroutine (summarize_async.go:271) — verified twice, independently, by injecting a panic. Cross-tenant isolation genuinely holds: cadf559 closes the client-supplied-session-id collision (tested against both per-tenant and a deliberately shared store), and OwnsKey (proxy.go:2107-2110) blocks cross-tenant marker restore. Structural prompt injection is blocked — a crafted reply can only ever become one user-role text message, so the PR #103 leaked-tool_use class is not reproduced. The episode panel's core arithmetic is correct: a hand-computed episode matches the live JSON to the last digit. The flight registry returns to empty on all four exit paths. The pinned-namespace memory leak I went looking for does not exist (TTL still applies; 20,000 sessions × 3 keys held at max).

---

Blockers

B1 — CacheAllows permits compacting a live prefix, and one false positive poisons the whole session · CONFIRMED (3 agents)

components/trigger.go:99-110, components/cachephase.go:30-34,72.

cachephase.go's load-bearing justification is that Unknown implies no live prefix: "on those deployments MaxCachedIdx stays -1 … There is no live prefix whose invalidation we would be avoiding." That implication is false. CacheRemaining returns ok=false when IdleMs <= 0, and apply.go:551 only sets IdleMs if prevAt > 0 && nowMs > prevAt. Three reachable states have MaxCachedIdx >= 0 with IdleMs == 0 → Unknown → permitted by cold, pre_expiry and the shipped pre_expiry_or_cold default:

(a) Two turns at the same clock instant — the deployed Tracker path, end-to-end through apply.BodyOpts:

turn2: maxCachedIdx=1 ttlMs=300000 idleMs=0 coldCache=false phase=unknown
=> the shipped cache_state default permits rewriting a prefix cached 0 ms ago

A backwards NTP step (nowMs < prevAt) widens that window from one millisecond to arbitrarily long.

(b) The legacy no-Tracker path (apply.go:552-554) sets maxCachedIdx from the store and never sets ttlMs/idleMs at all. The proxy and /compact both pass a Tracker (so the deployed service is not exposed by this one), but every library consumer of BodyFull/BodyOpts is. The comment at apply.go:490 naming /compact is stale.

(c) cache_mode: on on a non-Anthropic provider — maxCachedIdx=1, ttlMs=3600000, idleMs=0, phase=unknown, permitted.

summarize never reads MaxCachedIdx/TailOnly (zero grep hits across all three summarize files), so the cache-state gate is the only thing between it and a live prefix.

This is session-persistent, not per-turn. Once a checkpoint exists, the splice at summarize.go:332 runs before and independently of the gate check at :388 — Defect 3's fix working as designed. So a single falsely-Unknown turn commissions a summary against a live prefix and then rewrites the forwarded prefix for the rest of the session, warm turns included. Fix belongs in the gate; do not add a cache-state check at the splice or Defect 3 returns.

Suggested fix — one guard, root cause for all three paths:

func (t Trigger) CacheAllows(c *Ctx, p CachePhase) bool {
	// Unknown is only safe when there is provably no live prefix.
	if p == CachePhaseUnknown && c != nil && c.CacheAware && c.MaxCachedIdx >= 0 &&
		t.CacheState != "" && t.CacheState != CacheStateAny {
		return false
	}
	switch t.CacheState { ... }
}

Applied and regression-tested: go build ./... plus go test ./components/... ./apply/... ./config/... ./proxy/... ./dash/... — the only failures were the four bug-demonstrating tests inverting. Every existing test in the PR still passes, including cachephase_test.go's "cache_state %q refused an UNKNOWN phase" guard, whose fixtures all use MaxCachedIdx: -1 — which is why they never caught this.

Secondary fix worth doing regardless: IdleMs == 0 should mean "zero idle", not "unknown". Carry -1 for unknown from apply.go and guard on c.IdleMs < 0. A request with genuinely ~0 idle is the warmest possible cache and is currently classified Unknown.

B2 — coldByArithmetic inherits the exact keep-alive false positive its own docstring says it exists to avoid · CONFIRMED

components/trigger.go:127-130.

The function's entire justification is that Ctx.ColdCache must not be trusted, because proxy/keepalive.go never updates the turn tracker — the −$708 mechanism promexport.go:807 documents. It then computes CacheTTLMs - IdleMs, and IdleMs derives from the same un-updated prevAt that keep-alive never touches. On a kept-alive session with a real gap past the TTL it returns true and permits compacting the live prefix, under the shipped pre_expiry_or_cold default — i.e. it does precisely what it was written to prevent, on exactly the sessions someone is paying pings to protect.

This needs a design decision, not a patch: you cannot fix it by adjusting the arithmetic, because the input it trusts is stale for the sessions in question. Either keepalive.go updates the tracker, or the cold signal must not derive from prevAt at all. Both have costs; that is the author's call.

B3 — The summarizer's own model call is never debited on the async path, inflating reported net · CONFIRMED (static and live, independently)

dash/compactepisode.go:440 and :500 are the only readers of CGLLMCostUSD, and both are gated on isFreshSummary(r) != "". Commit f61e8c0 moved the detached call's cost off t0's row and onto t0+1's (cheapmodel.ReplayUsage + store.UsagePrefix), because the goroutine finishes after t0's row is written. t0+1 carries EventAwaitedCheckpoint, which :531 explicitly classifies as a replay. So the cost is written to the database and then silently skipped by the walk.

The repo already said this out loud, twice. f61e8c0's own message: "The episode panel charges that spend as a debit, so a missing cost inflates the reported net." proxy/cgllm_test.go:193: "TWO REQUESTS, and the cost lands on the SECOND." The proxy side was fixed; the dash side was never updated to read it from where it now lands.

TestAuditAsyncSummarizerCostIsNeverDebited
  summarizer cost = 0, want 0.005 (net = 0.1, overstated by the model call)

Fix (one line): move cur.SummarizerCostUSD += r.CGLLMCostUSD out of the isFreshSummary branch at :440 so every turn inside the span contributes it. That slightly over-attributes other components' cg spend (F8) — the conservative direction.

This leans the way this file repeatedly insists a savings figure must never lean, and it was invisible in the acceptance run because summarizer_cost_usd was never cross-checked against turn 18's raw cg_llm_cost_usd.

B4 — On [1m] model ids the frac gate can never fire, and Opus resolves 5× low · CONFIRMED

internal/modelinfo.normalize() strips a provider prefix but not the [1m] context-length suffix, so the id never matches a LiteLLM key and the chain falls through to DefaultStatic (exact=false). FracResolvable requires CtxWindowExact, and summarize.go:184 ships min_request_frac: 0.9. Resolved through the production chain with production's own prices.yaml:

model id	production chain	frac gate
claude-haiku-4-5	200,000 exact	can fire
aws/claude-sonnet-5	1,000,000 exact	can fire
aws/claude-opus-5	1,000,000 exact	can fire
aws/claude-sonnet-5[1m]	1,000,000 exact=false	never fires
aws/claude-opus-5[1m]	200,000 exact=false	never fires, and 5× low

None of production's 42 prices.yaml entries carries a window:, so Table.Window returns ok=false and the operator table cannot rescue it.

Two honest scope corrections. (i) On the gateway credential the test proxy used, the [1m] ids 403 ("team can only access aws/claude-sonnet-5, aws/claude-opus-5"), so no traffic on them reaches the gate there — but they are routable for other credentials on this host, and Claude Code is configured with exactly aws/claude-opus-5[1m]. (ii) The 5×-low Opus window is currently saved from being acted on by exact=false — but OutputFloor/IsHuge/extract_llm's frac use the non-exact Window(), where too-low is only safe in one direction.

Fix: strip a trailing [...] suffix in normalize(). A MODEL_PRICES entry with a trailing * is a working operator-side workaround in the meantime.

Also note the cold-start window: refreshIfStale fetches asynchronously, so round-1 lookups return exact=false even for models in the map. After every restart, summarize silently cannot fire until the map lands.

---

The empirical arm — the mechanism does not fire on real agent traffic

Real claude -p session, 14 turns, claude-haiku-4-5, shipped defaults (min_request_frac/cache_state absent so applySummarizeTriggerDefaults installed 0.9 / pre_expiry_or_cold):

0 of 14 turns fired, with both gates independently closed on every turn. Peak fill 150,840 = 0.754 of the window (threshold 180,000). cache_miss_reason was hit on 13 turns and cold_start on 1 — zero pre_expiry, zero ttl_expiry. Firing required scripting a deliberate 310 s idle gap after forcing billed input to 191k.

The PR concedes pre-expiry is hard to reach and argues expired is "the reachable half". On real traffic neither occurs, because an active agent keeps hitting the cache. The intended path appears unreachable in normal agent use.

The one forced episode lost money: −$0.0144, from provider usage blocks at production rates, including the summarizer's measured $0.0727. Break-even is roughly six more warm turns than the session had. acted was verified before crediting anything (verdict=acted, saved=139,297).

The dashboard reported net_usd = +$0.376 for that episode — wrong sign and wrong magnitude, from B3 plus a second cause: the credit is priced at the cache-write rate on turns that were cache hits (10.3× per turn).

The stall moved rather than disappeared. Firing turn: 0.378 ms. Next turn: 88,917 ms. The PR predicts "a typical episode carries ONE turn that stalls for a few seconds" and sets defaultSummaryWait = 120s for "the pathological tail, not the common case" — the first episode measured 89 s. Worth re-examining whether 120 s is the right cap and whether the wait honors the incoming request's cancellation.

Taken together: the acceptance run validated a path that normal usage does not take, and it used claude-haiku-4-5 — the one model class with no [1m] suffix, and therefore the only one where the feature could have fired at all.

---

Should-fix

Gating

• trigger.cache_state is unvalidated on 2 of the 3 components that accept it, and inert on both. trigger.go:97 claims constructors reject a bad value; only summarize does (summarize.go:246-250). extract_llm and extract accept cache_state: pre_expiryy without error — and neither consults the key (zero CacheAllows/CachePhase hits), so even a valid cache_state: cold is silently inert on a maximally warm turn. The settings form advertises it on all three with the hint "Restrict firing by the prompt cache's state". Fix: a shared Trigger.Validate() called by every constructor, and suppress both keys from TriggerFields for components that do not consult them.
• pre_expiry_seconds is unvalidated against the TTL. 600 on a 300 s TTL makes remaining <= preExpiry true for the entire lifetime → every warm request becomes PreExpiry → compaction every turn, on a config the form accepts silently. Fix: reject >= 300 (or clamp to a fraction of CacheTTLMs) where cache_state is already validated.
• Four readers of "is the cache cold", three disagreeing. coldByArithmetic drops the 60 s clock-skew margin apply.cacheIsCold applies to the same timestamps (apply.go:977-982, coldMargin at :884), so for a full minute of every session's expiry the gate says cold while apply says warm — converting the PR's "strictly better, unconditionally" case into the harmful one. This also refutes the framing at cachephase.go:81-85: the ColdCache-first check only makes the classifier more willing to say cold, and the reachable disagreement is the opposite one. All in a file whose comment says it exists so there is "one fact, one reader".
• Fires drops the frac gate entirely when PrevBilledInput == 0 (trigger.go:205). summarize is protected by pairing with FracResolvable; extract_llm.go:667 calls Fires without it, so the gate goes unapplied on turns with no prior billed input — firing more than main, the opposite of that call site's comment. Reach: theoretical for shipped presets (codesmart/house set only min_request_tokens: 3000), real via the settings form, which offers the key.
• Fires changed from max() to AND — a real semantic change, currently untested. No shipped config sets both thresholds, so no operator is affected today.
• Provider generalization. OpenAI is wrong in both cache modes: under auto, cacheAware=false → Unknown → permits compaction every turn, silently discarding OpenAI's real automatic prefix cache; under on, apply.cacheTTL's default: arm assumes a 1-hour TTL, so the gate never permits. No middle setting exists. A second hardcoded 5 min lives at dash/event.go:868. Bedrock and Vertex-fronted Claude are fine (both in the family switch; dispatch is on the provider argument, so the aws/ prefix never reaches it).

Async

• Latent data race: summarize_async.go:313 hands the live request Ctx to the detached goroutine, which reads c.Store via effectiveMode — contradicting the discipline stated 55 lines earlier ("Everything the goroutine needs, read HERE while we are still on the request's goroutine"). The race detector trips on a test that writes c.Store after Offload returns. No production write exists today (Ctx is per-request and never pooled), so this is latent — but it is exactly the shape where the next Ctx field write becomes a silent production race. One-line fix (hoist effectiveMode, pass mode into commitAsyncSummary) verified against all 8 audit tests under -race.
• The three new pinned namespaces saturate the store's pin budget. store/store.go:171-225 pins SumPrefix, BilledPrefix, UsagePrefix, all session-keyed — and cg:bin: is written on every response, not just summarizing sessions. Measured pinnedN == pinCap exactly (2,500 at defaults). Once saturated, admission silently declines, so newly-written cache-destructive entries (cg:frozen:, cg:result:) go unpinned and LRU-evictable — the loss pinning exists to prevent. store.go:100 already warns these share the budget. No metric exposes saturation.
• recover() is silent — no log, no counter.
• Goroutine exit depends on the Model client honoring ctx; an assumption, not enforced.

Dashboard

• A turn that closes one span and opens another is debited twice. :418 declares credited with the intended rule; :510 discards it (_ = credited). The existing test misses it twice over — its fixture uses miss(CacheTTLExpiry) so causedWriteUSD returns 0, and it asserts the new episode's cost without asserting the closing one is zero.
• The panel renders all zeroes, in green, in the normal state. open_net_usd, open_turns, voided_net_usd and the whole episodes[] drill-down are computed server-side and rendered nowhere in app.js (grep count: 0 each). An open span shows Our cost=$0.00 Net=$0.00 in good-text green while the real position is −$0.0025. 64d488f documents this state and points at a per-episode row that is not rendered — a false-green resolved by a doc change.
• The query plan collapses without sqlite_stat1. Same 50,000-request DB, same query: aborted at 11 m 14 s with no stats vs 0.078 s with them (50,000² ≈ 2.5×10⁹ index probes). janitor.go:82 runs PRAGMA optimize every 5 min so production usually has stats — exposure is the cold window plus the staleness case that comment itself warns about. A CROSS JOIN barrier does not help (SQLite cannot reorder a LEFT JOIN's operands); the problem is index choice. Measured fix: CREATE INDEX idx_rc_request_comp ON request_components(request_id, component) → 0.081 s, stats-independent.
• ?span=NaN / ?fill=NaN defeat queryFrac's guard.
• The dataset read is unbounded and reports no truncation; the route is uncached on a main tab and shares the KV-cache semaphore.
• The credit side inherits saved_usd's defects — saved_usd is the panel's only credit source (:765, :529-538) — while the debit side is computed independently from billed columns and correctly splits 5 m/1 h. The credit side structurally cannot split them: modelinfo.Price has no 1 h field. See saved_usd prices our own token count at the provider's rate, under-reporting savings by ~12% (measured) #240; cite the 12.4% content divergence, not the 3.38× (which is mostly scope, not part of the removed content).
• WindowUnknown excludes rather than mis-buckets, so no wrong number is published — but coverage goes to zero on a [1m] deployment.

Security (details in a separate pass; nothing cross-tenant)

• Unvalidated summary content. summaryWrapper (summarize.go:771) concatenates the cheap model's raw reply with no length cap, no marker stripping and no untrusted-content framing — then wraps it in authoritative instructions. Structural/role injection is blocked, so this is content-level steering from any page, file or command output the agent reads. Cheap fix: cap the reply against the span it replaces, and strip <<cg: / </summary> before wrapping (which also closes the forged-marker path, since the wrapper appends its own marker and one inside the reply is never legitimate).
• The detached goroutine holds a live tenant credential for up to 300 s after the request completes.

---

Docs / nits

• docs/reference/config.md states trigger.cache_state defaults to pre_expiry; the code ships pre_expiry_or_cold (summarize.go:185). The doc drops the half the PR body calls both "the reliable half" and "the reachable half". docs/components/summarize.md:91 has it right.
• No upgrade note: an existing summarize: {trigger: {min_request_tokens: 40000}} silently acquires min_request_frac: 0.9 and cache_state: pre_expiry_or_cold, going from "fires on every 40k turn" to almost-never (or never, per B4). There is no CHANGELOG.
• newContentBilled classifies on CacheRead > 0 rather than the cache verdict; an unpriced episode still publishes dollars.

Not attributable to this PR

Seven components/offload tests (TestGatePermitsHighReuseContent, TestBreakEvenSizesMatchTheDocumentedVerdict, and five more) fail on origin/main at -count=2 — pre-existing extract_econ global-state leakage, independently reproduced. Not a regression here, but it means the PR's "go test -race clean" claim holds only at -count=1, and it deserves its own issue.

Claims that could not be verified here

The EstimatorDivergence percentiles; Defect 2's narrative (the 300,128 ms hang against upstream_ms=2,838, the 5–14 s / 6–19 s comparators, turn 20's 400 prompt is too long); the acceptance-run figures; and the −$708 keep-alive figure. All are production measurements quoted from the repo or the PR body, internally consistent but not reproducible from the tree.

[OsherElhadad]

Follow-up: live-run results — one blocker now confirmed by observation, and a correction to B4's severity in my previous comment

The empirical arm finished after I posted. It changes two things, one in each direction. Total spend $1.5855, wall time ~1 h 05 min, claude-haiku-4-5, production's own MODEL_PRICES.

Correction — B4 is a latent resolver defect, not a live production blocker. My previous comment overstated it.

I wrote that the [1m] ids are "routable for other credentials on this host." That is not supported. Tested live, on the same gateway credential:

model id sent	gateway	proxy's ctx_window
claude-haiku-4-5	200 OK	200,000 — gate resolves
aws/claude-sonnet-5	200 OK	1,000,000 — gate resolves
aws/claude-sonnet-5[1m]	403	never reaches upstream
aws/claude-opus-5[1m]	403	never reaches upstream

team not allowed to access model. This team can only access models=[… 'aws/claude-haiku-4-5',
'aws/claude-sonnet-4-5', … 'aws/claude-sonnet-5', 'aws/claude-opus-5'].

So on the ids this gateway actually serves, summarize's frac gate does resolve — including on aws/claude-opus-5 at a correct 1,000,000, with window_not_exact clearing after turn 1. The static prediction that summarize "cannot fire on opus or sonnet here" is REFUTED for the routable ids.

What remains CONFIRMED is the underlying defect: normalize() does not strip a bracketed suffix, so wherever such an id is routed, the frac gate silently never fires and aws/claude-opus-5[1m] resolves to 200,000 against a real 1,000,000. That is worth fixing — a one-line change in normalize() — but it should be read as latent/defensive, not as a live outage on this deployment. Please downgrade B4 accordingly; the blocker label was mine and it was wrong.

B1 is now confirmed by live observation — and the reachable trigger is ordinary concurrency, not a clock coincidence

This is the strongest new result. FORCED config with cache_state: pre_expiry and min_request_frac: 0; under that setting a warm turn is declined with cache_state_declined_warm, and with every request ≤14 s apart on a 5-minute TTL, PreExpiry (which needs ≥240 s idle) is unreachable. So any turn the gate permits must have been Unknown. A 9-message session was warmed, then 6 rounds of 4 concurrent identical requests drove IdleMs to 0:

max_cached_idx across all 26 boundary lines:   25 x max_cached_idx=8    1 x max_cached_idx=-1
component runs:                               26
  declined by the cache gate:                 13
  PERMITTED by the cache gate:                13     <-- all necessarily Unknown

Observed: cache_aware=true, max_cached_idx=8, permitted as Unknown. Two concurrent turns are enough — and an agent issuing parallel sub-requests does that routinely, so this is not an exotic deployment or an NTP edge case. It reframes my "same clock instant" description as needlessly narrow.

Honest scope limit, from the agent and worth preserving: in this run the 13 permitted turns resolved to checkpoint replays (reused_checkpoint), which are byte-stable and harmless, and the single fresh summary paired with the max_cached_idx=-1 first turn where Unknown is legitimate. So what was observed is the gate opening over a live prefix, not a fresh summary destroying one. The gate opening is the defect; that no fresh summary landed on one of those 13 turns in a 26-request sample is luck, not protection.

A third, distinct dashboard defect: the credit is priced at the write rate on turns that hit

Separate from B3 (missing debit). Episode credit is priced at the cache-write rate ($9.5e-7) on turns whose cache hit, where the counterfactual is a cache read ($7.6e-8) — 10.3× per turn ($0.1323 vs $0.0129). Combined with B3, the panel reported net_usd = +$0.376 where the measured effect was −$0.0144. Wrong sign, and both causes attributable to a line: compactepisode.go:440 and creditTurn/Event.baselineDeltaUSD.

Note the two are independent, so fixing only B3 still leaves the sign wrong.

The rest, unchanged from my previous comment

• 0 of 14 turns fired on a real claude -p session under shipped defaults; peak fill 0.754 vs the 0.9 threshold; cache_miss_reason hit × 13, cold_start × 1, zero pre_expiry, zero ttl_expiry. Firing required a scripted 310 s idle gap.
• The one forced episode lost $0.0144 including the summarizer's measured $0.0727; break-even ≈ 6 further warm turns.
• Hot path genuinely free (0.378 ms); the stall moved one turn later — 88,917 ms on shipped defaults, against a design note predicting "a few seconds".

The SHIPPED and FORCED configurations were kept strictly apart throughout, and acted was verified before any delta was credited.

[OsherElhadad]

Final note: the episode panel is live on a real deployment — a reviewer hypothesis to the contrary is refuted

Closing a conflict between two of my reviewers, because it changes how urgent B3 is.

One pass argued the panel currently publishes nothing on this host: with MODEL_PRICES carrying no window: entries, Table.Window returns not-ok (table.go:202-209), so the chain would fall through to DefaultStatic's {"claude", 200000} catch-all — which is exact=false always — and compactepisode.go:322 would exclude every row into window_unknown. The inference drawn was that F1–F11 are invisible in production and only go live when someone adds a window:, which would neatly explain why none had been noticed.

That premise is wrong, and the live arm already contradicted it (aws/claude-sonnet-5 observed at ctx_window=1000000 with window_not_exact absent after turn 1). The missing step is that DefaultStatic is the last resort, not the first — LiteLLM answers ahead of it, and it reports exact=true on a per-model hit. Verified on this host:

$ curl -s -o /dev/null -w "%{http_code} %{time_total}s\n" \
    https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json
200 0.989s

$ grep -rn MODEL_INFO /etc/systemd/system/context-guru.service.d/
(no match)  ->  MODEL_INFO is not disabled; the map source is enabled by default

Egress to the map works, MODEL_INFO=off is not set, and normalize() reduces aws/claude-sonnet-5 to a tail that hits a published key. So the operator table's empty window: column is not load-bearing here — LiteLLM supplies both the window and the exactness before DefaultStatic is ever consulted.

The consequence runs the other way from the hypothesis: F1–F11 are live, not dormant. In particular B3 (the summarizer's own call never debited) and the write-rate-on-a-hit mispricing are affecting the Components tab on a running deployment today — which is why the net_usd = +$0.376 vs measured −$0.0144 result is a present-tense reporting error rather than a latent one. I'd treat B3's one-line fix as correspondingly more urgent.

Two smaller points from the same pass that stand on their own and are worth keeping:

• window_unknown short-circuits before the coverage block (:333-346), so an excluded deployment reports conversations: 0 as well — the panel cannot show the unflattering coverage figure it was partly built to expose. Real, and it would bite a deployment whose models genuinely miss the map (MODEL_INFO=off, an air-gapped host, or a gateway-private model id — including the [1m] ids if they were ever routed).
• A measured index fix: CREATE INDEX idx_rc_request_comp ON request_components(request_id, component) takes the no-statistics plan from aborted at 11 m 14 s to 0.081 s on a 50,000-request database, stats-independent. A CROSS JOIN barrier does not help here, since SQLite cannot reorder a LEFT JOIN's operands.

For the record on method: this is the second correction I've posted against my own review in this pass. Both came from checking a claim that happened to be convenient — the first that a defect was live when it was latent, this one that a panel was dark when it is lit.

[OsherElhadad]

Correction (3rd): the measured loss was understated 2.8×, the dominant defect is not the one I told you to fix first, and the [1m] reach is inconclusive

The empirical arm retracted and re-derived two of its own numbers after I had already posted them. All three items below supersede my earlier comments.

1. The measured net is −$0.03974, not −$0.0144 (n=1)

The original counterfactual used the compacted 22,032-token write as the counterfactual new tail, compounding the hypothetical uncompacted prefix to 213,856 tokens — above the model's own 200k window, i.e. physically impossible. Corrected to the real ~524-token tail and cross-checked two independent ways: −$0.03974 turn-by-turn, −$0.03986 by decomposition.

The loss is 2.8× larger than I published. Break-even was computed correctly and is unchanged at 6.09 warm turns.

I'm flagging the shape of that error as much as the number: a counterfactual that silently exceeds the model's context window is arithmetically detectable, and a plausibility check against the window would have caught it before it reached a reviewer. Worth a guard wherever this repo constructs a counterfactual prefix — including in the panel itself.

2. The write-vs-hit mispricing is 4.9× larger than the missing debit — I prioritised these backwards

Decomposed against the reported +$0.376:

Defect	Magnitude
Missing summarizer debit (B3)	$0.0727
Credit priced at cache-write rate on turns that hit	$0.3583

fix the debit only    ->  +$0.3033   still reports a PROFIT on a losing episode
fix the mispricing only ->  +$0.0177
measured truth          ->  −$0.03974

So my earlier framing — "treat B3's one-line fix as correspondingly more urgent" — was wrong on priority. B3 alone is not sufficient; it moves +$0.376 to +$0.303 and still publishes a profit on an episode that lost money. The mispricing is the dominant term and the one that flips the sign. Both need fixing, and if only one is done first it should be the rate, not the debit.

The rate defect remains two independent confirmations of the same line — the dashboard pass found it statically from creditTurn/Event.baselineDeltaUSD, the live pass found it on the wire — with the same mechanism and commit.

One honesty note on the decomposition: the two named defects sum to $0.4310 against a measured gap of $0.41574, leaving a −$0.0153 residue (3.7%) unaccounted for. So the decomposition is close but not exact, and there is likely a third smaller term. I'd rather state that than present the split as complete.

3. The [1m] resolver defect's reach is INCONCLUSIVE, and my credential premise was wrong

I wrote that the [1m] ids "are routable for other credentials on this host." That is now definitively refuted, not merely unsupported: the local settings file points at the same gateway host with the same key the test proxy loaded, so the 403 came from the identical credential — and it was a model-authorization refusal, with auth itself succeeding.

Since the client demonstrably works on that key with that model name, the client cannot be sending aws/claude-opus-5[1m] unchanged in the request body's model field. modelinfo reads the window from that field, so the resolver defect's reach is inconclusive and most likely latent-only — it needs whatever the body actually carries to be observed, which was not spent (~$0.01, one command, documented in the report's §1b).

Net effect: the normalize() fix is still correct and still worth making, but it should be treated as defensive hardening with no demonstrated production impact — not as a blocker. That is a further downgrade from my first correction.

---

Standing after three corrections

Unchanged and confirmed: 0 of 14 turns fired under shipped defaults (peak fill 0.754 vs 0.9; zero pre_expiry, zero ttl_expiry); the live observation of the CachePhaseUnknown gate opening over an 8-message cached prefix on 13 of 26 turns, reachable by ordinary concurrency; the free hot path (0.378 ms) with the stall relocated to the next turn (88,917 ms); and the panel being live rather than dark on this deployment.

Changed: the episode loss is 2.8× larger, the credit mispricing outranks the missing debit by ~5×, and the [1m] defect is latent with inconclusive reach.

Three of the corrections in this review have been against my own published claims, and each came from re-checking something convenient — a defect I called live that was latent, a panel I called dark that was lit, and a fix I called urgent that was the smaller half. The underlying findings survive; my confidence calibration on them did not.

[amiddavid]

Review addressed — with a scenario suite that reproduces two of your findings on live traffic, and one finding of its own

Thank you for this. Four blockers, three self-corrections, and an empirical arm that found the thing static review could not. Two of your findings I was able to confirm on the wire to the last digit, and the corrections you posted against your own claims made the triage materially better — I have followed your revised priority (rate before debit) rather than my own reading.

Every fix below was verified to fail with the fix removed.

---

The scenario suite is checked in — scripts/scenarios/

Your claude -p arm is the part of this review I most want to keep, so it is now three checked-in arms with a doc (docs/proposals/timely-compact-validation.md). Every arm builds its own proxy on its own port with its own dashboard db and a private CLAUDE_CONFIG_DIR; production Guru is not in the path.

export CG_SCEN_UPSTREAM="https://your-gateway.example.com"
export CG_SCEN_SRC=/path/to/a/frozen/checkout
tmux new -d -s scen 'scripts/scenarios/run-all.sh'

Arm	Config	Answers
A	shipped defaults, no injected idle	how often do both gates open on traffic nobody arranged?
B	forced, then three cold events in one span	does the cold credit accumulate per prevented rewrite?
C	forced, then warm only	is a warm turn's credit at the cache-read rate?

Two rig traps are documented in lib.sh because both produced silently empty runs: PATH being replaced rather than prepended (every turn exits 127 while the log just shows no rows), and a gap of 310s — past the TTL but inside ColdMargin — where the provider has already dropped the entry and the gate still, correctly, refuses to say so.

---

Your findings, confirmed on the wire

The credit mispricing — confirmed exactly, and it is a 12.5x ratio on identical content

Arm B, real Claude Code on claude-haiku-4-5. The same removal, on two consecutive turns that both hit:

row	verdict	saved_gross	saved_unique	saved_usd
#44	hit	123,251	123,251	$0.15406375
#45	hit	123,251	0	$0.01232510

0.15406375 / 0.01232510 = 12.5 exactly — same content, same verdict, priced 12.5x apart purely because the stash key was first seen on #44. That is the mechanism precisely as you described it: on the async path the goroutine has no Report, so the checkpoint key first reaches rep.CacheKeys on the turn that replays it, MarkUnique calls the whole removal new content, and unlike t0 that turn is credited.

creditTurn no longer inherits saved_usd. It prices the credit from saved_gross at the rate that turn's own verdict earns — creation where the entry lapsed, read where it hit. Nothing summarize removes inside a span is new content; that is what makes it summarizable, so the "unique at the creation rate" term has no meaning here. The bucket labels and the arithmetic now agree, where before a figure priced at the write rate sat in the bucket documented as "billed at the cache-read rate".

B3, the missing summarizer debit — confirmed live

Same run:

#43 (t0)  cg_llm_cost_usd = 0.000000   ttl_expiry   summary_started
#44       cg_llm_cost_usd = 0.008315   hit          reused_checkpoint

The cost is on t0+1, which isFreshSummary classifies as a replay, exactly as you said. Our model spend is now charged on every turn in the span. That slightly over-attributes another component's extraction call inside the span — the conservative direction, and the only rule that cannot silently miss our own call.

The double debit — fixed, and the fixture that hid it is fixed too

credited was declared with the rule written out above it and discarded as _ = credited. It is now structural (closing && opensNew), and costs go to the span the turn opens — that span's investment. You were right that the old test missed it twice over: its fixture used miss(CacheTTLExpiry) so causedWriteUSD returned 0. It now uses a hit, and asserts the closing span is charged neither cost.

---

B1 — the live-prefix gate. Fixed at two levels, and your concurrency framing was the better one

You were right that "same clock instant" was needlessly narrow and that ordinary concurrency reaches it. Two fixes, because your three reachable states have two different root causes:

IdleMs now distinguishes zero from unknown. CacheRemaining returned not-ok for a request arriving in the same millisecond as the previous one, and CachePhase turned that into Unknown. A turn with genuinely zero idle is the warmest cache there can be. IdleMs is negative for unknown, zero for zero; apply records nowMs == prevAt as the real zero it is; a backwards clock stays unknown rather than inventing warmth. That closes your case (a) at the root and classifies a concurrent turn as Warm, so the gate refuses it on the phase itself.

CacheAllows refuses Unknown when MaxCachedIdx >= 0, your suggested guard, scoped to callers that asked for a cache state. That covers (b) and (c), which genuinely have no readable TTL. cachephase.go's claim that Unknown implies no live prefix is corrected in place rather than left standing.

Your point that this belongs in the gate and not at the splice is in the code as a comment, with the reason: a gated turn must still replay or it reverts to the full transcript at the worst moment. That is also why one false positive was session-persistent.

Every existing guard in cachephase_test.go used MaxCachedIdx: -1 — the one value that makes Unknown safe. That is why none of them caught it, and the new cases say so.

On the cold margin — I tried your "one fact, one reader" fix and a test stopped me

You were right that coldByArithmetic dropped apply's 60s allowance. My first attempt put the margin in CachePhase so both readers shared one threshold, and TestSweepFiresOnlyInThePreExpiryWindow failed — correctly. The two readers answer different questions and the safe error runs opposite ways: extract_llm_sweep needs an entry that still exists (so assume it's gone), summarize's gate needs one that is certainly gone (so assume it may live). CachePhase therefore keeps calling nominal expiry Cold, and CertainlyColdByClock carries the margin for the compactor. The window between them is a deliberate dead zone where compaction declines — the honest answer for a window where we cannot tell.

Arm B measured what that costs: at 310s idle on a 300s TTL the provider had already dropped the entry (zero cache read, a full 183,660-token rewrite) and the gate still refused. Real conservatism with a real price.

B2 — not fixed, and no longer implied to be — #243

You said this needs a design decision rather than a patch, and I agree. CertainlyColdByClock's docstring now says so directly instead of leaving the impression the keep-alive case is handled. The issue records both options with their costs, plus a conservative interim: refuse keepalive + cache_state: cold on the same tenant.

B4 — fixed as defensive hardening, at your revised severity

normalize() strips a trailing [...]. Written up as latent with no demonstrated production impact, citing your own two corrections, because your final position on reach is the right one to record.

---

Arm A — the firing rate, and it is a sharper result than either of us had

53 turns, shipped defaults, real Claude Code on haiku, no injected idle:

peak fill                0.996     <-- the gate needs 0.900
turns at or over the fill gate   12 of 53
cache verdicts           {'hit': 50, 'cold_start': 1, 'unknown': 1, 'prefix_change': 1}
TURNS THAT FIRED         0 of 53
gates (summed)  {'below_request_trigger': 40, 'cache_state_declined_warm': 51, 'window_not_exact': 2}
inter-turn gaps (s)      min=0.1 median=7.3 max=44.3
  gaps >= 240s (the idle pre_expiry needs)   0 of 52
CLIENT COMPACTED         yes — client's own ceiling 199,184 = 0.996 of the window

Your 0-of-14 reproduces, and the diagnosis is not the one either of us assumed. The fill gate is not the obstacle: the session passed 0.9, 12 turns qualified, and Claude Code let the context reach 99.6% of the window before compacting. So "the client caps below our gate" is refuted on this model and version — which also means the min_request_frac: 0.5 in my earlier acceptance config was a test lever justified by a wrong reason.

The binding constraint is cache_state_declined_warm on 51 turns: pre_expiry needs ~240s of idle and the largest gap in a working session was 44s. An active agent keeps hitting its own cache. My acceptance run fired because I scripted a 394s gap; nothing in normal use produces one.

So the shipped default is, in practice, gated on a person stepping away for five minutes and coming back to a nearly-full context. That is a real and arguably valuable case — it is exactly when a cold rewrite of a 199k prefix costs the most — but it is not a general win and should not be argued for as one. docs/reference/config.md now carries these numbers instead of its previous claim that the default was pre_expiry.

This is the decision I am handing back rather than making: lower the fraction, widen pre_expiry_seconds, or accept the feature as a narrow safety net and document it as one. The mechanism is not broken; whether the default earns its place is a judgement.

---

Arm B's own finding — the cold bucket is empty at the shipped span, for a second reason

Three forced cold events inside one span. The credit accumulates and hand-checks exactly:

EPISODE (span=0.25) turns=7  cold=0.46219125  read=0.03697530  cg=0.00831500  net=0.49085155
hand                         3 x 123,251 x 1.25e-06 = 0.46219125   ✓
                             3 x 123,251 x 1.00e-07 = 0.03697530   ✓

Each cold event re-created a compacted prefix (48,967 / 66,107 / 66,216 tokens) where an uncompacted session would have re-created ~175,361 — ~115,000 tokens of prevented rewrite per event, three times over.

At the shipped span=0.10 the cold credit is $0.00, because the episode closes after two warm turns — 20,095 of a 20,000 target — and those two turns took ~30 seconds. A cold event needs ~6 minutes of idle. So a cold event essentially cannot fall inside a 10% span on a session whose turns append 16k-token tool outputs.

This is a second, distinct reason the headline bucket is structurally near-empty. #234 already fixed one ("cumulative spend closes the span in one turn"); this one is "cumulative new content closes it in two or three, which is half a minute of wall time". The 10% figure came from the original specification and it is measured in the right units now — it is simply short relative to the timescale a cache expiry lives on. Worth deciding alongside the firing-rate question.

Also visible on those cold turns: below_request_trigger and reused_checkpoint together. The gate declined and the turn replayed anyway — the gate-without-returning fix, on live cold turns, which is precisely where reverting to the full transcript would cost the most.

---

The false-green panel — you were right, and my earlier commit was the wrong fix

open_net_usd, open_turns and voided_net_usd were computed and rendered nowhere. The table showed a settled $0.00 in good-text green while the committed position was negative, and 64d488f documented that state in prose instead of fixing it. Arm B is the demonstration: at span 0.25 the settled total is $0.00 and the real position is +$0.49, entirely inside open_net_usd.

The Net cell now renders an em-dash when nothing in a group has settled, and a new exposure line reports the open spans, their turns and their net, with voided reported apart and never averaged in.

---

Also fixed

• ?span=NaN defeated the range guard — every comparison against NaN is false. NaN and Inf rejected first.
• The measured index. idx_rc_request_comp added, in additiveDDL rather than behind a version bump, because a bump discards every request row to add an index that changes no row's shape. Your 11m14s → 0.081s measurement is in the DDL comment, including that a CROSS JOIN barrier does not help and why.
• pre_expiry_seconds validated against the shortest cache lifetime, with a test that first demonstrates a one-second-old entry really does classify PreExpiry under a 600s window.
• Trigger.Validate shared by all three constructors, so CacheAllows' docstring stops promising something one of three did.
• max() → AND pinned in a test. You were right that a real semantic change had nothing asserting it.
• The detached goroutine no longer reads the live Ctx — effectiveMode hoisted, so it holds no Ctx at all.
• recover() is no longer silent — a counter and an ERROR line carrying the panic value.
• summaryWrapper sanitizes the reply: both marker spellings expand accepts, including the JSON-escaped one rawMarkerRe exists for, plus </summary>. Deliberately not attempted: detecting instructions (undecidable, and a filter that half-works invites reliance on it) or capping length (never-worse already reverts a grown turn, and truncation makes a long legitimate summary misleading).
• An unpriced episode no longer publishes dollars. summarizer_cost_usd is a stored figure independent of the model's rates, so it was the one field that survived. The rendered panel masked it; a JSON consumer of this public route did not.
• docs/reference/config.md said the default was pre_expiry. Corrected, with arm A's numbers.

Filed rather than fixed here

• #243 — B2, the keep-alive clock the cold gate trusts.
• #244 — the pin budget saturates silently and no metric says so. Your pinnedN == pinCap measurement is in it, with the argument that cg:bin: is the cheapest namespace to lose and should probably not hold a slot.
• #245 — OpenAI wrong in both cache modes, in opposite directions. Includes your verification that Bedrock and Vertex are fine, because the model-id prefix makes it look as though they might not be.
• #246 — the seven -count=2 failures on main. You were right to check main before attributing them.
• #247 — the inert cache_state on extract/extract_llm. Both obvious fixes break an existing contract, and both dead ends are written into the issue and into Trigger.Validate's docstring: hiding the keys fails the field-declaration contract, refusing the value fails the form contract. The real fix is splitting Trigger, which is a config-shape change with a migration.
• #240 — saved_usd's ruler. Arm B measures it again: 123,251 removed tokens by our count against ~115,000–126,000 by the provider's. Your steer to cite the 12.4% content divergence rather than the 3.38x scope figure is what the panel's new credit_quantity_note says.

Still open

The span width and the firing rate are now both measured, and neither is a code defect. They are arguments about the shipped defaults and they are the repo owner's call. Formal approval remains blocked by the shared account.

🤖 Generated with Claude Code