Skip to content

agent-core: compaction summary candidates (head-tail, bridge, state-sections, short, carry-all, validate), behind --compaction-candidates - #240

Closed
jaredLunde wants to merge 5 commits into
mainfrom
jared/summary-candidates
Closed

jaredLunde wants to merge 5 commits into
mainfrom
jared/summary-candidates

Conversation

@jaredLunde

Copy link
Copy Markdown
Contributor

Six compaction-summary candidates from the summary research, each its own commit and its own --compaction-candidates name. All off by default; with none named, every request, summary and persisted session is byte for byte what main produces.

Name What it does Evidence it answers
head-tail The re-render keeps a long tool result's first 500 and last 1,500 characters, plus up to 30 test-tally or error lines from the elided middle, instead of only its first 2,000. 66% of long test results in the eval had their pass/fail tally only after character 2,000, so the summarizer never saw it.
bridge The spliced summary message opens with "Checkpoint of earlier work. Messages after it are newer and win where they differ. Work listed as Done is on disk: don't redo or re-verify it unless something newer contradicts it." in place of the "Compacted from N tokens" line. The count moves to CompactionProvenance::tokens_before, which the session store prefers; previous_summary and the export strip the bridge, so it never stacks. Codex, opencode and the other major harnesses add such a line; nothing told our model how the checkpoint relates to the verbatim messages after it.
state-sections Adds ## Tests & Errors (latest) and ## Failed approaches / don't retry before Key Decisions, and turns Next Steps into one ## Next Step that must follow from the state ("Verify and finish" allowed). On the shipped template (cached instruction under either wire's framing, and the re-render) via shape_default_format, and on the state template. Only 55% of eval summaries mention a test tally; 20% of compactions re-fail the same command within 30 calls; summarized agents often fail to stop.
short A ~700-word target with per-section caps. Median summary length grew 771 → 2,167 words by round five; the evidence on length disagrees, so it is measured.
carry-all Every genuine user message rides <user-requests> (each ≤ 8,000 chars, ≤ 32,000 in all; over budget the first plus the most recent), instead of the first and latest three. Summaries keep ~17% of user constraints against 93–99.7% when the user's words are re-appended verbatim; Codex carries them all.
validate After the summary is final and before the carry is appended, deterministically appends what it lacks: the latest test tally and command (when the kept tail has none), the user's constraint lines, and a next step. A cheap, deterministic version of judge-based summary repair (+2.6 to +8.8 points in the literature).

Includes an end-to-end compact test for bridge + validate, plus unit tests per candidate.

Eval results (probe-based replay + continuation) are reported separately; nothing here changes defaults.

🤖 Generated with Claude Code

https://claude.ai/code/session_01JimHGjsfk2Ktm5GxyZJKKk

jaredLunde and others added 5 commits October 12, 2026 03:19
…ehind head-tail

A re-render cuts every tool result to its first 2,000 characters, and test runners print their
tally last: 66% of the eval's long test results had it only after character 2,000. Under the
`head-tail` compaction candidate a long result keeps its first 500 and last 1,500 characters, plus
every test-tally or error line of the elided middle (at most 30). Off by default.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JimHGjsfk2Ktm5GxyZJKKk
…n count, behind bridge

Nothing told the model how the checkpoint relates to the verbatim messages after it, and the
"Compacted from N tokens" line it did see is metadata only an export reads. Under the `bridge`
compaction candidate the summary message opens with "Checkpoint of earlier work. Messages after it
are newer and win where they differ. Work listed as Done is on disk: don't redo or re-verify it
unless something newer contradicts it." instead. The count rides the provenance
(`CompactionProvenance::tokens_before`), which the session store prefers to parsing the line;
`previous_summary` and the export strip the bridge so it never stacks. Off by default.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JimHGjsfk2Ktm5GxyZJKKk
…nd a short variant, behind state-sections and short

Two changes to the summary format, each its own candidate, on the shipped template (the cached
instruction under either wire's framing and the re-render alike, via shape_default_format) and on
the state template. `state-sections` adds `## Tests & Errors (latest)` (the last test command,
tally, failing IDs and current errors, verbatim) and `## Failed approaches / don't retry` ahead of
Key Decisions, and turns Next Steps into one Next Step that must follow from the state, "Verify
and finish" allowed: only 55% of eval summaries mention a test tally, 20% of compactions re-fail
the same command within 30 calls, and summarized agents often fail to stop. `short` sets a
~700-word target with per-section caps: median length grew 771 -> 2,167 words by round five, and
the evidence on length disagrees. Off, the shipped text is byte for byte unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JimHGjsfk2Ktm5GxyZJKKk
The request carry keeps the first and the latest three user messages at 4,000 characters each,
dropping whatever the user said in between. Under the `carry-all` compaction candidate it keeps
every genuine user message, each up to 8,000 characters and together up to ~8K tokens; over budget
the first is kept and the rest fill from the most recent backwards, as Codex does. Summaries keep
~17% of user constraints against 93–99.7% when the user's words are re-appended verbatim.
Off by default.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JimHGjsfk2Ktm5GxyZJKKk
…ep to the summary, behind validate

Only 55% of eval summaries mention a test tally and 34–40% kept the "don't edit tests" constraint.
Under the `validate` compaction candidate, before the carry is appended, the summary gets what it
lacks appended deterministically, never rewritten: the latest test tally and its command (when the
kept suffix shows none), the user's constraint lines from the request carry, and a next step when
it has none. The zero-cost version of the judge-and-repair loops that gain 2.6–8.8 points in the
literature. Off by default.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JimHGjsfk2Ktm5GxyZJKKk
@jaredLunde
jaredLunde enabled auto-merge October 12, 2026 10:31
@jaredLunde
jaredLunde added this pull request to the merge queue Oct 12, 2026
@jaredLunde
jaredLunde removed this pull request from the merge queue due to a manual request Oct 12, 2026
@jaredLunde

Copy link
Copy Markdown
Contributor Author

Closing: no candidate met its ship rule in the probe eval (bridge, short and validate dropped; sections, head-tail and carry-all inconclusive or untested). Not landing dead flags.

@jaredLunde jaredLunde closed this Oct 12, 2026
@jaredLunde
jaredLunde deleted the jared/summary-candidates branch October 12, 2026 15:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant