Skip to content

agent: a stronger model can write the compaction summary, behind --compaction-summarizer-model - #242

Closed
jaredLunde wants to merge 1 commit into
jared/summary-candidatesfrom
jared/summary-summarizer-model
Closed

jaredLunde wants to merge 1 commit into
jared/summary-candidatesfrom
jared/summary-summarizer-model

Conversation

@jaredLunde

Copy link
Copy Markdown
Contributor

Stacked on #240 (base jared/summary-candidates); it retargets to main once #240 merges. Needs #241 for a Sonnet summarizer to work in direct mode.

--compaction-summarizer-model <MODEL> (env AI_AGENT_COMPACTION_SUMMARIZER_MODEL, on run and serve) has another model write every compaction summary, e.g. claude-sonnet-5-5 for a Haiku or gpt-6-luna session. Swapping only the summarizer moved SWE-bench Verified 6.5 points in one study, and a weak summarizer did worse than no summary in another.

  • The summarizer gets a transport of its own, built the way a subagent's is (Agent::with_summarizer), because credentials and routing are model-keyed.
  • Every summary is then a re-render: only the session's own model can read its prompt cache, so the cached call is skipped.
  • The request runs without the session's thinking and effort settings, which are model-specific.
  • In serve, if the summarizer's client can't be built, a warning is logged and the session's model keeps summarizing.
  • The Compacted event's usage is the summarizer's, so cost accounting must price it at that model's rates.

Unset, nothing changes. It's a plain flag rather than a --compaction-candidates name because that set only holds on/off values.

Eval results (probe-based replay + continuation) are reported separately; nothing here changes defaults.

🤖 Generated with Claude Code

https://claude.ai/code/session_01JimHGjsfk2Ktm5GxyZJKKk

…ompaction-summarizer-model

Swapping only the summarizer moved SWE-bench Verified 6.5 points in one study, and a weak
summarizer did worse than no summary in another. `--compaction-summarizer-model <MODEL>` (`run` and
`serve`) has that model write every summary over a transport of its own, built the way a
subagent's is (credentials and routing are model-keyed): `Agent::with_summarizer`. Every summary is
then a re-render, since only the session's own model can read its prompt cache, and it runs without
the session's thinking and effort settings, which are model-specific. Unset, nothing changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JimHGjsfk2Ktm5GxyZJKKk
@jaredLunde

Copy link
Copy Markdown
Contributor Author

Closing: the stronger-summarizer arm was inconclusive (+0.056, CI crossed 0) at ~40x the per-summary cost. Not shipping a dead flag.

@jaredLunde jaredLunde closed this Oct 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant