feat(kvcache): ship a fixed reuse model, and put the budget arm on the page

[OsherElhadad]

What this adds

A fixed, checked-in reuse model, and with it the keepalive-budget arm on the KV-cache page.

PR #212 built BudgetPolicy and then could not show it to anyone. That was structural, not an
oversight: the arm is built from a Predictor, no Predictor existed outside a test, and the
registry's promise is that every name in it resolves. A name resolving to an arm with a nil
Predictor would have been worse than no name at all — Decide returns write_5m when it has
no opinion, so that arm never pings, and the page would have shown a flat write policy under a
label promising a learned one.

kvcache.ReuseModelV1	the model: a frozen discrete-time survival fit, ~30 coefficients, compiled in
kvcache/reusemodel.go	the arithmetic that reads it and answers Predictor
kvcache/registry.go	keepalive-budget joins Registry(); NewKeepAliveBudget is why it can
deploy/harbor/kv_ttl_keepalive_policy.py	--db/--model-out/--go-out — the fit wiring its own docstring said was missing
kv_ttl_reusemodel_drift_test.go	the guard the existing one structurally cannot be

dash/ needed almost nothing. KVCacheArms() probes the registry rather than carrying a
list, so the checkbox, the baseline <select> and the per-(user,hour) suggest sweep all gained
the arm for free. The only dash change is adding it to the default arm set, plus the name
constant.

The model, and the three constraints that pick its shape

It is a logistic regression, not the gradient-boosted pair fit() uses, and it gives up 0.087
of dollar-AUC to be one. That is the price of shipping, and all three reasons are real:

• Reviewable. Thirty-odd coefficients can be read in a diff, so a re-fit that broke the model
  would be visible. Thirty-seven trees of thresholds cannot be, so nobody could tell.
• Fixed. A model re-fitted at startup makes yesterday's dashboard figure unreproducible, and
  the page's whole claim is that replaying a historical window gives the same answer every time.
• Cheap on the request path, because the stacked PR has the live pinger consulting the same
  model, where proxy/keepalivestrategy.go's own standing rule is "a rule, or … a portable
  logistic-regression dot product — never an embedded model or a call out of the hot path."

Fitted on 18.4 days of the hosted capture: 219,650 idle spans, 37,418 of which reach the first
sweep — that second number is the population the decision is actually faced on, and it is worth
having in the provenance. Holdout AUC 0.9065 by row, 0.7837 by dollar, fitted on the early 70%
and scored on the late 30%; split by time, never at random, because one span contributes up to
max_k person-periods and a random split scores a row against its own siblings.

What it is worth

Replayed through the page's own scorer, against the fixed-5m baseline the page uses.

arm	pings	off the bill	of the ceiling	net per ping
optimal (unreachable — reads the future)	1,041	8.054%	100.0%	3.069
keepalive-budget, cap 5	8,790	2.065%	25.6%	0.0932
keepalive-budget, cap 8	9,003	2.059%	25.6%	0.0907
keepalive-5m flat k=5	59,139	1.860%	23.1%	0.0125
keepalive-budget, cap 2 (page default)	7,573	1.850%	23.0%	0.0969
keepalive-5m flat k=6	69,864	1.817%	22.6%	0.0103
keepalive-5m flat k=2 (DefaultMaxPings)	25,586	1.808%	22.4%	0.0280
keepalive-5m-once	13,538	1.516%	18.8%	0.0444
keepalive-5m flat k=8	90,779	1.381%	17.1%	0.0060
stop-reason-gated	14,969	0.710%	8.8%	0.0188
historical-probability	0	−0.128%	—	—

On this window the arm beats every reachable alternative on money — which is not what #212
measured. That PR reported a flat constant winning by 0.89 pp and said so plainly. Two things
changed and only one of them is the corpus: the window is longer (18.4 days against 16.9), and
three specification defects were fixed. Read #212's figure as measuring a differently-specified
model, not as a result this one contradicts.

The budget varies per conversation, which is the whole point and the one thing a flat MaxPings
cannot do — 41.4% of conversations are worth no ping at all and the arm declines them, which
is where the 6.6× reduction in ping volume comes from:

budget	0	1	2	5	6	8
share of decisions	41.4%	31.9%	22.5%	1.2%	3.0%	0.0%

Three specification defects, each a trap worth not re-entering

All three are in the doc, because all three produced plausible numbers and no error.

1. Collinear sweep features. sweep_k, log_elapsed and log_secs_to_deadline are exact
functions of the sweep index — the third is life - interval at every sweep, a constant. A tree
ensemble tolerates that as redundancy; a linear model cannot. The first fit produced −4.77 and
+4.09 on two features that move together, summing to a flat ~14% hazard at every sweep where
the corpus measures 4.86% at the first falling to 0.24% by the seventh. The arm bought its
maximum budget on 97% of decisions and saved less than a flat MaxPings=2. The sweep index
is a one-hot now — a free baseline hazard per sweep — and the shape it recovers rises from −1.25
to +1.83 and then falls, i.e. not monotone, which is exactly why assuming monotone was wrong.

2. The last sweep's label. person_periods labels survival at k == max_k against inf, so
it is identically 0. Harmless as a truncation; fatal once the curve is composed into a CDF —
s_max_k fits to ~0, F jumps to 1.0 at the tail, and Windows computes
h_max_k = (1−F)/(1−F) = 100%, a certainty the backward induction propagates back through every
earlier sweep. Every span then buys the maximum budget. gap_s > t_max_k + interval is observable
for every span in the frame, so nothing needed censoring.

3. cache_ttl is a train/serve skew, but only once the model is wired in. Offline it is the
tier the corpus recorded and fitting on it is self-consistent — which is why the reference fit is
fine. At serve time the seam hands over Observation.TTL, and simulate.go:488 sets that from
st.tier: the tier the arm under test chose on the previous turn. The feature means one thing
in training and another in serving, and the second is the model's own prior output fed back as an
input. Dropped, for 0.0001 of holdout dollar-AUC.

user_id is dropped too, for a different reason: a per-tenant dummy generalises to no tenant the
fit never saw and would put tenant identifiers in the shipped binary, while stat_p/log_stat_n
already carry that tenant's own history in a non-identifying form.

The fit weight: an argument that lost to a measurement

Fitting unweighted should be right, and the argument is a good one: the arm compares a
probability against the rates' own 8.70% break-even, so what it needs is calibration, not ranking,
and a stake-weighted fit is deliberately miscalibrated — it reports the hazard of a dollar, not
of a conversation, and the two differ fourfold here (21.10% stake-weighted at the first sweep
against 4.86% by row). log_prefix is a covariate precisely so the size effect survives without
the weighting.

Both were fitted and both were replayed through the real arm:

--weight	pings	off the bill	net per ping	budget 0 on
stake	9,003	2.059%	0.0907	41.4%
none	7,250	1.836%	0.1004	56.4%

stake ships, because money is the question the page is asked. What the calibration argument
predicted did not happen: being better calibrated did not produce better decisions, it produced
fewer of them. --weight none stays selectable and the argument stays in the code, because the
0.22 pp gap is within shouting distance of this corpus's ±0.42 pp seed noise — it is decided, not
settled.

Tests

kvcache — 10 new, each asserting one property the arm depends on rather than one code path:
the CDF is a distribution across every horizon from before the first knot to past the last;
F(t_1) = 0 by construction, so Windows does not condition on reaching the first sweep twice; a
horizon past the grid clamps rather than extrapolating past 1.0; an unseen model level falls to the
catch-all and not to zero (zero is not "unknown", it is "exactly the reference level"); the level
is case-folded, because the corpus spells one upstream two ways; the standardisation is actually
applied, so the coefficients are not being read against raw values; ok=false is reserved for "no
model"; Validate refuses a table naming a term this build cannot score, which would otherwise be
applied to a silent zero; the unfitted max_k+1 sweep coefficient cannot change a budget; and the
registry's arm really does carry ReuseModelV1 and really does implement PingBudgeter.

deploy/harbor — the guard the existing one cannot be. --fixture feeds a hand-written step
CDF, so the coefficients, the feature derivation and the standardisation are all outside it — its
own docstring says "the drift guard covers NEITHER". --score closes that: it drives the
checked-in reusemodel_v1.json, the artifact the Go table was generated from, so a hand-edit of
the generated file and a regeneration that was never committed both fail. Its fixture carries
raw Observation fields, never derived features, so a log1p applied to milliseconds on one
side and seconds on the other is caught rather than agreed on. Four layers compared — per-sweep
survival, the CDF on and off the knot grid, the per-window h/s, and the budget — at 1e-9
absolute, because both sides do the same arithmetic in the same order and anything above float
noise is a real divergence rather than a tolerance question. Ten cases, including a model id the
fit never saw, the same id differing only in case, a 5× rate scale, and a MinPrefix gate.

TestUnbudgetedArmsAreUnaffectedByTheSeam becomes an allow-list rather than "none". SILENTLY
was the load-bearing word in its message and it still is. Naming the set also catches the other
direction — an arm dropping off the budget path and quietly running on the flat cap — which an
assertion of "none" never could. Its cross-replay isolation check is untouched.

What this does not change

DefaultMaxPings stays at 2. That still caps the arm below the 8 sweeps it reasons over, and it
wins there anyway (1.850% against flat k=2's 1.808%, on 7,573 pings against 25,586) — the doc
says plainly what the cap costs and how to lift it. No route is added. No live behaviour changes:
the model is consulted by the simulator only. Wiring it into the live pinger is the stacked PR,
where the gate-versus-budget question and the ping-volume argument belong.

One thing observed and deliberately not changed: BudgetPolicy.Predictor's doc says a nil
predictor means "every budget falls back to Config.MaxPings". The fallback is real in
Simulate, but it is unreachable through Decide, which returns write_5m on ok=false — so a
nil-predictor arm fires no pings at all rather than falling back to the cap. Left alone as #212's
to correct.

Verification

go vet ./...                                        clean
gofmt -l .                                          clean      (generated file included)
go build ./...                                      clean
go test ./...                                       pass
go test -race ./kvcache/ ./deploy/harbor/ ./dash/   pass       (dash 479s)
mkdocs build --strict                               clean
kv_ttl_keepalive_policy.py --self-test              pass