Offer provider-native compaction as an alternative to summarize, for deployments with no summarizer model

[amiddavid]

summarize is the only compaction mechanism this proxy has. Anthropic also ships server-side compaction, and for some deployments it is a better fit than ours — notably any deployment with no summarizer model available, where summarize currently gates on no_model and does nothing at all.

This is a per-deployment alternative, not a runtime fallback. Both paths decide before the request goes upstream, so nothing falls through from one to the other on failure; and native compaction exists only on Claude, so the llm-d and self-hosted deployments that most need a ceiling cannot use it.

The provider-side facts, so they need no rediscovering

Researched in docs/proposals/timely-compact.md (§"What 'native' compaction means"), which has the full detail:

• Beta header compact-2026-01-12. Available on all five platforms (1P, Claude Platform on AWS, Bedrock, Vertex, Foundry).
• Requested per call:

{"context_management": {"edits": [{
  "type": "compact_20260112",
  "trigger": {"type": "input_tokens", "value": 400000},
  "pause_after_compaction": true
}]}}

• The trigger is a threshold, not an imperative: there is no "compact now". Minimum 50,000, default 150,000. A proxy chooses when by choosing value per request, which is enough to place it wherever our own gate would have fired.
• pause_after_compaction: true returns stop_reason: "compaction" with the summary instead of the model's answer.
• The resulting compaction block must be echoed back on subsequent requests.

What actually makes this more than a config knob

Storing the block is not the hard part, and an earlier framing of this issue wrongly said it was. This proxy already keeps per-session state under pinned namespaces — cg:sum: (summarize's checkpoint), cg:len:, cg:ttl:, cg:seen:, cg:bin:, cg:use:, plus content stashes — and already re-emits stored bytes verbatim on every turn to keep the cached prefix stable. An opaque provider block held per session and echoed is the same shape as the summary checkpoint we already replay. Losing it degrades the same way too: the transcript is re-sent in full and the provider re-compacts, which costs money rather than correctness.

The three things that are genuinely different:

1. It breaks the reversibility invariant

CLAUDE.md: "Every lossy Offload must be reversible (a <<cg:HASH>> marker + the stashed original in the Store)." The pipeline enforces it — dropping content without a stash is refused unless the component sets rep.Irreversible.

We cannot stash what the provider removes. The compaction happens server-side, so there is no span for us to hash and store, and context_guru_expand / GET /expand cannot restore it. So the native path is structurally irreversible in a codebase whose central guarantee is that compaction is undoable.

That is the decision this issue exists to force, and it is a policy question rather than a coding one:

• treat it as a deliberate Irreversible mode, the way marker_mode: summary|off already is — legitimate, but it must be conspicuous in config and on the dashboard, because an operator enabling "native compaction" would not expect to lose expand;
• or refuse to offer it where reversibility is required, and document it as available only to deployments that opt out of expand.

2. Two editors of one history

If native compaction is enabled alongside our own components, the provider and the pipeline are both rewriting the prefix. Our components reason about a cached boundary (MaxCachedIdx, the volatile-tail split, the frozen-decision replays) that assumes we are the only party editing. Whether native is exclusive of the rest of the pipeline, or merely of summarize, needs deciding before it ships.

3. The pause round-trip — real, but avoidable

With pause_after_compaction: true the provider answers with the compaction instead of the model's response, so the proxy must recognise stop_reason: "compaction", store the block, and re-issue the request with it echoed — an extra upstream call the client never asked for, inside a request loop that already handles expand rounds and SSE splicing.

Without the pause, the provider compacts and answers in one turn, and the block arrives beside a normal response. Then it really is just store-and-echo. So this complexity is opt-in, and the first implementation should probably decline it.

Suggested shape

• A mode on the compaction path — summarize stays the default; native is selected explicitly, per deployment or per model.
• Native sets the per-request trigger.value from the same arithmetic the cache-aware gate uses, so "when to compact" is one decision with two executors.
• Start without pause_after_compaction.
• Named on the dashboard as a distinct mechanism, since its episodes are not comparable with ours: no saved_usd, no stash, no replay.

Why it is worth having

The case that motivates it: a deployment with no summarizer model. Today summarize gates no_model and the transcript grows until the provider rejects it. Native compaction needs no model of ours, no credential, and no CHEAP_MODEL — the provider does the work. That is a real hole in the current design, and it is the strongest argument for this issue.

Context

Researched and then deliberately descoped during #234, on the owner's direction: "for now I dont want the 0.99 gate, we can just get to 100 and claude would do its native compaction. just implement a new trigger for summarize." The design that shipped assumes the client caps the context (Claude Code compacts as it approaches its budget) and positions our summarizer below that ceiling. This issue is the phase-2 option that assumption leaves open, for the deployments where no client ceiling exists.