Mark a Herdr badge whose numbers the transcript has outrun, and rescan to fix it

[lis186]

摘要

Herdr 側邊欄徽章的數字只更新到 ccxray 最後記錄的那一輪。ccxray 沒在觀測的 session（在 proxy 外啟動，或 hub 消失後續跑）會讓 transcript 繼續長、索引卻停住，於是徽章把一直在跳的 age 配上凍結數小時的 context%。

本 PR 讓徽章在磁碟上存在我們沒看到的 turn 時明示 stale、收回顏色宣稱，並觸發一次節流、加鎖、分離的重掃來修好它。

判定訊號經過兩次修正：檔案 mtime 不能用（Claude Code 會寫沒有對應 API request 的 metadata 記錄），實測 41 次發作有 37 次是誤報。改用 core 自己的 turn 定義後降到 4 次、0 誤報；tail 視窗放大後 8 次、仍 0 誤報。

What changed

Piece 1 — the marker. A session is stale when its Claude transcript holds a COMPLETED turn newer than the newest turn ccxray logged, using core's own rule (server/importer.js:165-172: an assistant record with usage, a non-zero token total, a parseable timestamp). Reading that timestamp also puts both sides of the comparison on the same clock — it is the field the importer stores as receivedAt.

When it fires, ctx_bar's signal slot reports stale 11h, the summary appends the same, ctxBand drops to unknown, and the ctx token gets a stale suffix. The percentage itself stays.

Two things it deliberately does not use:

• Elapsed time alone. A finished session and a user who stepped away are equally quiet, and their numbers are still correct. Over the full index an elapsed-only rule fires on 4467 of 4470 sessions.
• The transcript's file mtime. Claude Code appends system, last-prompt, mode, permission-mode and file-history-snapshot records with no API request behind them, so mtime advances on a session ccxray is watching perfectly. Measured over 161 locatable real sessions, an mtime rule fired on 41 — 37 of them false, including both sessions that first looked like proof of the bug.

Piece 2 — the rescan. ccxray import --once is a throttled (10min), lock-guarded, single-shot transcript scan the badge fires detached. It does NOT refuse while a hub is running, unlike rebuild-index --reimport: that refusal is right for a destructive full rebuild and would leave every hub user permanently stale here. What made the refusal necessary is avoided instead — this process only appends index lines and never writes sessions.json, because session-index.js's tmp file has a FIXED name, so tmp+rename protects against a crash but not against a second writer. ADR 0019 records that contract; the hub re-derives via the existing "index.ndjson newer than sessions.json" rebuild.

Evidence

Detection, on the real index. 194 locatable sessions in a 60MB slice; the detector fires on 8, and every one verifies against an independent signal — a 100%-imported index whose transcript holds 63 to 639 more completed turns than ccxray ever logged. Zero false positives. One is a session that was running on this machine at review time, 308 turns behind.

Coverage, disclosed rather than glossed. About 40% of sessions have no locatable transcript (codex panes, a cwd ccxray never recorded) and are never marked. The bias is deliberately toward silence.

Differential, taken in throwaway worktrees per docs/verification-principles.md rather than by checking files out over a dirty tree:

• baseline origin/main in a sanitized env (env -i): 2157/2157, 4 runs, 0 flakes
• origin/main + only the new test files: 12 behavioural tests fail, the guards pass
• the lock race test on 44989a0: fails at round 0 with two owners; passes on 560a16f
• branch: 2184/2184, twice, with an empty CCXRAY_HOME

Test isolation. Staleness reads a scan root outside CCXRAY_HOME (the ADR 0015 R4 class) and runs in-process, so a throwaway $HOME is not available. Every call site pins CCXRAY_IMPORT_HOMES; an instrumented run confirms reads of the developer's real ~/.claude* went 7 → 0. docs/testing.md records the requirement and the fixture shape.

Review gate

codex was unavailable (usage limit, resets 2026-08-20). Per owner instruction the gate ran on grok, which returned DON'T-SHIP on the lockfile.

It was right. wx-else-unlink-and-retry admits two owners: both racers see the same stale lock, both unlink, the second removes the first's fresh lock. The first fix — hardlink for atomic create, inode identity for ownership — was still wrong, and the new 8-racer test found two owners in round 2 under load: stat and unlink are separate syscalls, so R1 can stat the stale inode, R2 can replace it with its own live lock in between, and R1's unlink removes R2's. The takedown is now serialized by a second hardlink, and the residual (a process dying inside the reclaim) is documented at the site.

Also from that review, each fixed here: the mtime gate rested on a false claim about mtime never preceding file content (a restore or a touch breaks it in the dangerous direction); the 512KB tail window hid true positives (4MB now — firings went 4 → 8, all verified); importOnce now refuses an injected env it cannot honour; transcriptSlug normalizes // and a trailing /; the slug test built its fixture with the production regex so it would have passed for any rule; --force had lost its dedup coverage to a lock probe.

The ccxray import --once CLI is new and user-facing; plugins/herdr/README.md and the CLAUDE.md architecture table document it, along with CCXRAY_BADGE_STALE_MS and CCXRAY_BADGE_IMPORT_DISABLE.

🤖 Generated with Claude Code

---

Correction — the race test was not evidence

The fix(herdr): close the lock race commit claimed its 8-racer test fails on the pre-fix lock "at round 0". It did, twice. Re-running it on an idle machine gives three consecutive passes on the buggy code; 8×2, 8×3, 12×2 and 16×1 all pass too. The interleaving it depends on needs contention to appear, so its sensitivity tracked machine load — the runs that "proved" the fix happened while a wedged suite and eight unkillable orphan processes were competing for CPU.

It was also expensive in a way that showed up as someone else's failure. Measured in a herdr pane, this branch failed 2 unrelated process-timing e2e tests per run (a different pair each time) against 2157/2157 on origin/main under identical conditions. Shrinking the test to 6×2 made it cheap and completely blind.

Replaced by an assertion on the mechanism rather than the symptom: plant a stale lock, plant a .reclaim held by a live pid, require that acquireLock neither acquires nor touches the stale lock — with a companion test that the same state minus the live reclaimer does proceed, so the guard is a gate and not a permanent refusal. The pre-fix lock has no .reclaim concept and reclaims regardless: 3/3 FAIL on 44989a0, 3/3 PASS after, with no timing dependence.

Pre-merge review (fable) — items addressed

• origin/main had moved two commits ahead and the branch conflicted on CLAUDE.md. Merged; the conflict resolves to the rewritten architecture table (which already carried the import-once.js row) plus the ADR 0019 invariant line. Verified nothing of the concurrent export-sync.js work was swept in — the merge commit touches it not at all.
• Comments citing superseded measurements: the tail-bound discussion still said 512KB and "61 of 161" seventeen lines below a 4MB constant, and evidenceStaleness's header still said "4 of 161 / 192 to 752" while its own body said 194. Each now states the bound it was measured at.
• filesSkipped → duplicatesSkipped: scanAndImport's skipped counts already-indexed turns, not files, and this is a machine-readable CLI field.
• Dropped evidenceStaleness and WATCHDOG_MS from module exports — nothing imports them.
• ADR 0019 now records the case the env guard does not cover: a caller passing process.env itself is allowed through and keeps the flush guard set on its own process.

Full suite on the merge result: 2185/2185, empty CCXRAY_HOME.

Rendered-sidebar verification (owner, 2026-08-17)

Done, as an A/B with a negative control rather than a single confirming look. A
synthetic session was pushed to one pane twice, the two runs differing only
in the timestamp inside a throwaway transcript, so a marker that fired regardless
of staleness would have shown up identically in both:

	ctx row as drawn	summary row as drawn
A transcript level with the index	33% · claude-opus-4-6 · $1.23	opus-4-6, 4.0h, $1.23
B transcript 4h ahead of the index	33% stale · claude-opus… · $1.23	opus-4-6, 4.0h, $1.23 …

A shows no marker, B does. The summary differs too — B is truncated because
· stale 4.0h was appended past the column width.

That also corrects something stated earlier in this PR. The summary was described
as unreachable on a sidebar whose rows do not name $summary; it reaches the
screen anyway, because refresh-badges.js passes it as every state's
--state-label, and that config's first row does name state_text. Two of the
three channels are therefore live on a minimal layout, not one.

The verification touched no real data and needed no reinstall: fixtures were
temp dirs, CCXRAY_BADGE_IMPORT_DISABLE=1 suppressed the rescan, and the tokens
carried a TTL so herdr dropped them on its own.

Still open

• The ctx_bar colour claim is unverified. ctx_band is asserted to flip to
  unknown at the token layer and install-sidebar-summary.js maps that to
  #a6adc8 against #a6e3a1 for green, but the sidebar used for verification
  does not reference any $ctx_bar* token, and rewriting a hand-written config
  to see one colour was not judged worth it.
• codex review gate not run (usage limit until 2026-08-20). grok stood in per owner instruction and returned DON'T-SHIP on the lock; that finding is fixed above.