daemon: serialize transcript metric events incrementally - #2246
Draft
scouredimage wants to merge 2 commits into
Draft
daemon: serialize transcript metric events incrementally#2246scouredimage wants to merge 2 commits into
scouredimage wants to merge 2 commits into
Conversation
Transcript batches were bounded by event count only (1000), and read_jsonl_line buffered lines of unbounded length. A transcript whose events embed file contents (normal for agent tool results) could put hundreds of MB into a single batch, which downstream redaction and metrics conversion amplify several times over -- ballooning daemon RSS past the memory watchdog within seconds (git-ai-project#2244). - read_jsonl_line: cap a single line at MAX_JSONL_LINE_BYTES (8 MiB). The read is byte-based (read_until), NOT read_line: the cap can slice a multi-byte character, and read_line's UTF-8 validation would return InvalidData instead of classifying the line as Oversized -- wedging the stream at a fixed watermark. Oversized lines are skipped without being buffered (skip_until); callers advance their watermark past them via the new JsonlLineState::Oversized state. Non-UTF-8 content within the cap keeps read_line's InvalidData contract. - All JSONL byte-offset stream parsers (claude, codex, copilot, cursor, droid, gemini, pi, windsurf): stop a batch early once MAX_BATCH_BYTES (8 MiB) of raw JSON has been accepted; remaining events arrive in later batches. Not covered here: amp, continue_cli, and opencode parse whole files into a DOM (serde_json::from_reader) and need a separate treatment -- called out in git-ai-project#2244 as follow-up. Part 1/7 of the git-ai-project#2244 fix stack.
store_metrics_in_db materialized a Vec<String> of every re-serialized event while the full Vec<MetricEvent> (each holding the redacted JSON tree) was still alive -- two complete copies of a transcript batch resident at once (git-ai-project#2244). The stream worker now serializes each MetricEvent as it is built and drops the tree immediately; persistence takes the pre-serialized rows via the new persist_metric_jsons_blocking. Existing callers of persist_metrics_blocking are unchanged (it now delegates to the same insert path). Behavior note: previously one unserializable event failed the whole batch (all-or-nothing insert); the stream-worker path now drops the failing event with a warning and persists the rest. These are best-effort diagnostics, and partial persistence beats losing the batch. Part 2/7 of the git-ai-project#2244 fix stack (stacked on 1/7).
This was referenced Aug 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part 2/7 of the #2244 stack.
store_metrics_in_dbmaterialized aVec<String>of every re-serialized event while the fullVec<MetricEvent>(each holding the redacted JSON tree) was still alive - two complete copies of a batch resident at once. The stream worker now serializes each event as it is built and drops the tree immediately.Behavior note: previously one unserializable event failed the whole batch; the stream-worker path now drops the failing event with a warning and persists the rest (best-effort diagnostics).
Stack
Each PR targets
main; the incremental diff of part N is its last commit. Root-cause analysis, production evidence, and the repro live in #2244.Verification
daemon memory limit): unpatched v1.6.24 aborts in ~1 s and re-aborts after respawn
(watermark 0); with the full stack the daemon survives, peak RSS 76 MB vs 589 MB,
zero memory emergencies, traced-git probes at 0.05-0.07 s throughout.
cargo fmt --checkclean;cargo test --lib: 2,401 passed. Three failures(
commands::upgrade::...pending_update,daemon::...conflict_resolution_note_read_errors...,git::authorship_traversal::...ai_touched_files...) reproduce identically on cleanmainin this environment - pre-existing, unrelated.