Skip to content

fix(opencode): keep the session container off the recursive watch budget - #1355

Merged
wesm merged 3 commits into
kenn-io:mainfrom
rodboev:pr/1208-opencode-watch-topology
Aug 9, 2026
Merged

fix(opencode): keep the session container off the recursive watch budget#1355
wesm merged 3 commits into
kenn-io:mainfrom
rodboev:pr/1208-opencode-watch-topology

Conversation

@rodboev

@rodboev rodboev commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

OpenCode's SQLite container, its WAL, and its storage archive share one recursive watch unit over the configured root, so the live container draws on the same shared recursive budget as the archive. A root reached after that budget is spent registers no native watch at all, and exhaustion inside the root's own walk marks the container's coverage degraded along with the archive's.

This separates each OpenCode-family root into a shallow container unit covering the database and its WAL and a recursive unit at <root>/storage. A shallow watch never draws on the recursive budget, so the container's coverage no longer depends on the archive's size. The split applies only where it is representable: a root with no storage directory keeps the single recursive unit, because naming an absent watch root would plan a polling obligation probed on a path that may never appear, and a probe that cannot be satisfied defers every other obligation on the same configured dir. A symlinked root also keeps it, since the daemon refuses to watch a recursive root through a symlink and gates the configured dir's reconciliation on the link target instead.

A native watch shared by overlapping roots is now charged to the shared budget once. RecursiveWatchResult.Watched still counts the directories a root covers; Allocated counts the watches a registration installed, and registration subtracts that. Reuse is settled before the budget check, so a root whose directories are already watched registers cleanly against an exhausted budget and records its ownership, and reclaiming a shared watch returns one budget slot rather than one per root that walked it.

Two units share one configured root and the engine dispatches every changed path once per emitted watch root, so unitScopeAllows scopes classification to the units of the root that owns the path; without it a WAL write would run the shared container's whole session listing twice. Configured roots can nest, so the deepest one containing the path owns it, and a virtual member path resolves to its physical container before the check. The container unit still claims storage paths: whether the storage unit exists is a filesystem fact, while the engine resolves each provider's watch roots once and reuses that set, so deferring would leave a storage tree created afterwards claimed by nothing. That costs one repeated source lookup per storage event, which the engine's per-pass source set discards; the database fan-out cannot double, because a storage watch root never contains the database or its WAL.

This PR stands alone and supersedes PR #1318, whose conditional storage-unit split covered the same provider topology. Portable lifecycle ownership of a missing watch root is not attempted here. The bounded parser feed that motivated PR #1331 is abandoned rather than shipped: OpenCode exposes no producer-owned cross-drain journal cursor, so that work was closed and this slice is the salvage.

Refs #1208

@roborev-ci

roborev-ci Bot commented Aug 6, 2026

Copy link
Copy Markdown

roborev: Combined Review (6c1783a)

Watcher changes need fixes for three medium-severity coverage and budgeting issues.

Medium

  • internal/sync/watch_backend_fsnotify.go:139 — Shared native watches are still charged against the registration budget. Overlapping roots can prematurely exhaust the budget and force unrelated roots into polling because existing watches increment Watched, and budget exhaustion is checked before watch reuse.

    • Fix: Track newly allocated native watches separately from logically covered paths, subtract only physical allocations, and permit an existing watch to gain another owner when no allocation budget remains.
  • internal/sync/watch_backend_fsnotify.go:443 — Pending recursive-root activation uses covering ancestors as recursive watch owners rather than the pending root. Exclusions are consequently evaluated relative to the ancestor, potentially skipping the new root while reporting successful activation, disabling polling, and missing descendant changes.

    • Fix: Install the recursive subtree using the pending path as its owner; retain coveringOwners only for lifecycle tracking.
  • internal/sync/watch_backend_factory_darwin.go:370 — Darwin logical roots are deduplicated by path alone, despite shallow and recursive units now being valid at the same path. If the shallow unit registers first, recursive coverage may never be installed and nested changes can be silently missed.

    • Fix: Key logical roots by path and recursion mode, or upgrade duplicate-path registrations to recursive coverage while merging scopes.

Reviewers: 2 done | Synthesis: codex, 12s | Total: 7m36s

@roborev-ci

roborev-ci Bot commented Aug 7, 2026

Copy link
Copy Markdown

roborev: Combined Review (2c07e1c)

Watcher lifecycle changes have two medium-severity reliability issues; no security vulnerabilities were found.

Medium

  • cmd/agentsview/main.go:2650 — A missing optional storage root creates a probe that blocks polling for the entire configured OpenCode root. If the container later appears with only SQLite, or its shallow watch fails, SQLite changes are never polled. Avoid creating a blocking missing-root obligation for optional storage units; rely on the parent container’s polling obligation when lifecycle ownership is unavailable.

  • internal/sync/watch_backend_fsnotify.go:435 — Pending-root handling occurs after os.Stat. Broken symlinks or temporarily inaccessible directories return early without transferring coverage to polling, potentially leaving the root permanently unwatched. Detect pending paths before Stat, transfer non-watchable creations to probe-gated runtime polling, and retain lifecycle tracking.


Reviewers: 2 done | Synthesis: codex, 7s | Total: 14m4s

@roborev-ci

roborev-ci Bot commented Aug 7, 2026

Copy link
Copy Markdown

roborev: Combined Review (4526eae)

High-severity build break and medium-severity stale-session risk remain.

High

  • internal/sync/watch_backend_factory_darwin.go:1252appendFallbackPollPlan now requires path, probe, and scopes, but this call omits the probe, breaking Darwin builds. Pass ObligationProbe(state.plan.Path, state.plan.Optional, state.plan.Exists) before state.plan.Scopes.

Medium

  • internal/sync/watch_backend_fsnotify.go:481 — Reactivating a pending root only enumerates current files without authoritative reconciliation. Sessions removed while the root was unavailable can remain stale. Queue an acknowledged reconcile-root event upon lifecycle activation, and release polling only after reconciliation succeeds.

Reviewers: 2 done | Synthesis: codex, 10s | Total: 7m37s

@roborev-ci

roborev-ci Bot commented Aug 7, 2026

Copy link
Copy Markdown

roborev: Combined Review (a084781)

Code review found two medium-severity correctness issues; no security vulnerabilities were identified.

Medium

  • internal/sync/watch_backend_fsnotify.go:805completeRootActivation removes the activation marker before polling is released. During this lock gap, a concurrent create can see the root as degraded and ownerless; polling may then be released, leaving the new subtree unwatched. Validate the generation, release polling, clear degradedRoots, and remove the activation atomically under watchMu.

  • internal/sync/watch_backend_factory_darwin.go:1072 — An initially absent optional root that appears as a symlink enters polling without setting plan.Exists. Its obligation therefore retains an empty probe; if the symlink later breaks, polling may reconcile the parent as empty and tombstone its sessions. Mark the optional unit as having existed when lifecycle handling observes it before installing its polling obligation, including fallback-recovery paths.


Reviewers: 2 done | Synthesis: codex, 11s | Total: 8m39s

@roborev-ci

roborev-ci Bot commented Aug 7, 2026

Copy link
Copy Markdown

roborev: Combined Review (dfdd275)

High-severity watcher lifecycle issues remain and could incorrectly tombstone archived sessions.

High

  • cmd/agentsview/main.go:827 — A fresh collection cannot distinguish an optional root that never existed from one that disappeared. Recovery and polling may authoritatively reconcile the remaining container and tombstone archived storage sessions.
    • Fix: Track whether optional roots were ever observed or contain archived sources. Defer missing observed roots, omitting the probe only when absence is proven harmless.

Medium

  • internal/sync/watch_backend.go:142 — A root created after collection but before its covering fsnotify watch is installed can be marked lifecycle-owned without being registered or producing a future create event, permanently suppressing polling.
    • Fix: Atomically arm pending ownership and recheck the path, immediately activating an existing directory or retaining polling coverage.

Reviewers: 2 done | Synthesis: codex, 10s | Total: 5m6s

A recursive registration counted every directory its walk visited against
the shared watch budget, including directories an earlier root already
watched natively. Overlapping roots spent budget on kernel watches the
process was already paying for, and a later root could be refused coverage
that already existed.

RecursiveWatchResult now reports Allocated alongside Watched: Watched stays
the count of directories the root covers, Allocated counts the native
watches the registration installed, and RegisterRoots subtracts Allocated.
Reuse is settled before the budget check, so a root whose directories are
already watched registers cleanly against an exhausted budget, and its
ownership is recorded so removing the first root does not drop coverage the
second still needs. Watch cost is recorded once per native watch rather than
once per registration, so reclaiming a shared watch returns one budget slot
instead of one per root that had walked it.

Refs kenn-io#1208
@rodboev
rodboev marked this pull request as draft August 8, 2026 00:46
@rodboev
rodboev force-pushed the pr/1208-opencode-watch-topology branch from dfdd275 to bbed1ef Compare August 8, 2026 00:46
@rodboev rodboev changed the title fix(opencode): preserve watch coverage through root lifecycle fix(opencode): keep the session container off the recursive watch budget Aug 8, 2026
@roborev-ci

roborev-ci Bot commented Aug 8, 2026

Copy link
Copy Markdown

roborev: Combined Review (bbed1ef)

No issues found.


Reviewers: 2 done | Synthesis: codex | Total: 3m58s

@rodboev
rodboev force-pushed the pr/1208-opencode-watch-topology branch from bbed1ef to fe91f6a Compare August 8, 2026 01:11
@roborev-ci

roborev-ci Bot commented Aug 8, 2026

Copy link
Copy Markdown

roborev: Combined Review (fe91f6a)

Medium finding: shared-root deduplication can still leave supported OpenCode configurations without a native watch.

Medium

  • cmd/agentsview/main.go:2525 — Watch-root deduplication promotes a shared OpenCode shallow container root to recursive when another provider uses the same path. If the recursive budget is exhausted, the container receives no native watch. Preserve the mandatory shallow registration separately from recursive coverage, or otherwise guarantee the shared root is registered outside the recursive budget.

Reviewers: 2 done | Synthesis: codex, 6s | Total: 5m27s

rodboev added 2 commits August 7, 2026 21:52
…e units

OpenCode's SQLite container, its WAL, and the storage archive shared one
recursive watch unit over the configured root, so the live container drew
on the same shared recursive budget as the archive. A root reached after
that budget was spent registered no native watch at all, and exhaustion
inside the root's own walk marked the container's coverage degraded along
with the archive's. The plan now emits a shallow container unit at the
root, covering the database and its WAL, and a recursive unit at
<root>/storage. A shallow watch never draws on the recursive budget, so the
container's coverage no longer depends on the archive's size.

The split applies only where it is representable. A root with no storage
directory keeps the single recursive unit: naming an absent watch root
would plan a polling obligation probed on a path that may never appear, and
a probe that cannot be satisfied defers every other obligation on the same
configured dir, leaving it with neither native coverage nor polling. A
symlinked root also keeps it, because the daemon refuses to watch a
recursive root through a symlink and gates the configured dir's
reconciliation on the link target instead, and that check reads the unit's
Recursive flag.

Two units share one configured root and the engine dispatches every changed
path once per emitted watch root, so unitScopeAllows scopes classification
to the units of the configured root that owns the path; without it a WAL
write would run the container's session listing twice. Configured roots can
nest, so the deepest one containing the path owns it, and a virtual member
path resolves to its physical container before the check. The container
unit does not defer storage paths to the storage unit: whether that unit
exists is a filesystem fact, while the engine resolves each provider's
watch roots once and reuses that set, so a storage tree created afterwards
would be dispatched only against the container root and claimed by nothing.
An empty watch root stays unscoped for callers that do not dispatch per
watch root.

Non-OpenCode plans and the Darwin backend keep their current contracts.

Refs kenn-io#1208
The daemon keeps one watch root per path, so two providers naming the same
directory merge into a single root that is recursive if either of them was.
A shallow registration installs its watch unconditionally, but a recursive
one walked the whole tree under the shared budget and installed nothing at
all once that budget was spent, so merging a shallow unit into a recursive
one left the shared directory with no native watch.

The root's own directory is now the mandatory part of a recursive
registration and its subtree the discretionary part, which is what a
shallow registration already does for the same path. An exhausted budget
still reports BudgetExhausted for the subtree and hands it to polling.

A mandatory watch installed past the budget stays outside the accounting
entirely, the way a shallow root's already does: it is not charged, so
removing it cannot refund a slot the process never spent and leave headroom
above the cap for runtime subtree adds to claim.

Refs kenn-io#1208
@rodboev
rodboev force-pushed the pr/1208-opencode-watch-topology branch from fe91f6a to 93f6486 Compare August 8, 2026 01:52
@roborev-ci

roborev-ci Bot commented Aug 8, 2026

Copy link
Copy Markdown

roborev: Combined Review (93f6486)

No issues found.


Reviewers: 2 done | Synthesis: codex | Total: 6m58s

@rodboev
rodboev marked this pull request as ready for review August 8, 2026 02:15
@wesm
wesm merged commit 14992d3 into kenn-io:main Aug 9, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants