Repository navigation
agent: a stronger model can write the compaction summary, behind --compaction-summarizer-model - #242
Closed
jaredLunde wants to merge 1 commit into
Closed
jaredLunde wants to merge 1 commit into
jaredLunde wants to merge 1 commit into
Conversation
…ompaction-summarizer-model Swapping only the summarizer moved SWE-bench Verified 6.5 points in one study, and a weak summarizer did worse than no summary in another. `--compaction-summarizer-model <MODEL>` (`run` and `serve`) has that model write every summary over a transport of its own, built the way a subagent's is (credentials and routing are model-keyed): `Agent::with_summarizer`. Every summary is then a re-render, since only the session's own model can read its prompt cache, and it runs without the session's thinking and effort settings, which are model-specific. Unset, nothing changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JimHGjsfk2Ktm5GxyZJKKk
Contributor
Author
|
Closing: the stronger-summarizer arm was inconclusive (+0.056, CI crossed 0) at ~40x the per-summary cost. Not shipping a dead flag. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #240 (base
jared/summary-candidates); it retargets to main once #240 merges. Needs #241 for a Sonnet summarizer to work in direct mode.--compaction-summarizer-model <MODEL>(envAI_AGENT_COMPACTION_SUMMARIZER_MODEL, onrunandserve) has another model write every compaction summary, e.g.claude-sonnet-5-5for a Haiku orgpt-6-lunasession. Swapping only the summarizer moved SWE-bench Verified 6.5 points in one study, and a weak summarizer did worse than no summary in another.Agent::with_summarizer), because credentials and routing are model-keyed.serve, if the summarizer's client can't be built, a warning is logged and the session's model keeps summarizing.Compactedevent'susageis the summarizer's, so cost accounting must price it at that model's rates.Unset, nothing changes. It's a plain flag rather than a
--compaction-candidatesname because that set only holds on/off values.Eval results (probe-based replay + continuation) are reported separately; nothing here changes defaults.
🤖 Generated with Claude Code
https://claude.ai/code/session_01JimHGjsfk2Ktm5GxyZJKKk