perf(desktop): merge archived observer pages a page at a time

[Chessing234]

Refs #5985 — the "archive path is unbounded" half, taken as a cost problem rather than a retention-policy one.

appendArchivedChannelEvent merged one event at a time, and each call did a linear dedup scan of the whole channel window and then re-sorted it. Ingesting a page of P events into a window of M therefore cost O(P x M) comparisons plus P full sorts. The archive window is deliberately uncapped — MAX_OBSERVER_EVENTS protects the live per-agent store only — so M grows with the age of the channel, and paging back gets slower the further back you go. That is the transcript-window lag the issue describes on otherwise idle hardware.

ingestArchivedObserverEvents now groups a page by (agent, channel) and merges each group once: one pass to index the existing window into a Set, one to filter, one sort.

Measured, same workload both ways (25 pages of 200 events into one channel, ending at a 5000-event window), via the repo test loader on this machine:

	total
before	7287 ms
after	78 ms

Scope, deliberately. This changes only how a page is merged — not what is retained, not the cap, not the SQL. The issue's second half (retention of kind 24200 telemetry that converts to a metric 0.76% of the time) is a product decision about what to keep, and the deep-history separation between the live and archive windows is load-bearing by design, so neither belongs in the same change. If the archive read should also carry a LIMIT, that is worth doing on top of this — it is a smaller win once the merge is linear, and it changes what the user can scroll back to.

Tests — 4 new cases pinning what the batch merge must keep doing:

• a newest-first page (the order SQLite returns) lands ascending;
• duplicates are dropped both within one page and against the already-loaded window — the per-event version got the within-page half for free by appending to the window each time, so it is the easy thing to lose;
• one page spanning two channels keeps the two windows separate;
• a 1-event older page merged into a 2000-event window neither drops nor reorders it.

The 31 existing tests in ingestArchivedObserverEvents.test.mjs pass unchanged, which is the main safety argument — the security guards and drop rules on that path are untouched.

Verified locally in desktop/: pnpm typecheck, pnpm check (2 warnings + 2 infos, all pre-existing on main), pnpm test 4958 passed, git diff --check. Not run: the app itself, so the user-visible lag improvement is inferred from the measurement above rather than observed in the UI.

[themiguelamador]

The page-at-a-time merge itself is sound and preserves the tested behavior. I found one minor documentation issue in the touched path:

• P3 — Update the archive-ingest contract after changing the implementation. The function docs still said channel-scoped events route through appendAgentEvent, claimed a live-delivered event is skipped by archive ingestion, and a nearby merge comment still named the removed singular helper. The stores are deliberately separate; cross-store dedup occurs in mergeObserverEventWindows, while this helper now performs archive-local batching/dedup. Leaving the old contract here would misdirect the next change in a subtle stateful path.

Fix branch: https://github.com/Complear/buzz/tree/review/pr-6065-fix
Commit: d1315b388

Validation: the full desktop suite passed (4,958 tests), along with desktop typecheck, full desktop checks, and git diff --check. The check output contains only the four pre-existing advisory Biome diagnostics outside this PR.

[Chessing234]

Confirmed and fixed in 2d517a1. (Complear/buzz 404s, so this is written from your description.)

All three doc claims check out, and I verified the load-bearing one rather than taking it on trust: archiveEventsByChannel is written from exactly one place, appendArchivedChannelEvents, which only the archive-ingest path calls — so no live event is ever in the window to dedup against, and "archived events that are already present (live-delivered) are silently skipped" was never true of the new code. Cross-store dedup is mergeObserverEventWindows, at read time.

Fixed:

• ingestArchivedObserverEvents no longer claims it "routes through appendAgentEvent" — it now describes the actual split (channel-scoped → batched appendArchivedChannelEvents; channelId-less → appendAgentEvent) and says the dedup is archive-local.
• appendArchivedChannelEvents had inherited the same wrong sentence; it now states it dedups both against the window and within the incoming page.
• mergeObserverEventWindows no longer names the removed singular appendArchivedChannelEvent.

One thing you didn't list that I hit while making the change: the batch commit inserted archiveDedupKey between the merge doc block and the function it documents, so the whole contract was attached to the key helper and the merge function had none. Split them.

Verification: desktop typecheck, biome, and the focused ingest suite (35 passed).