Skip to content

feat(map-efficient): parallel-wave worktree merge coordinator (Part of #284 Phase 2)#297

Merged
azalio merged 1 commit into
mainfrom
feat/284-wave-worktree-parallelism
Jun 27, 2026
Merged

feat(map-efficient): parallel-wave worktree merge coordinator (Part of #284 Phase 2)#297
azalio merged 1 commit into
mainfrom
feat/284-wave-worktree-parallelism

Conversation

@azalio

@azalio azalio commented Jun 27, 2026

Copy link
Copy Markdown
Owner

Wires the existing wave/DAG scheduler to per-subtask worktree isolation (Phase 1, #296) so a parallel wave of independent subtasks each runs in its own worktree and is accepted atomically via a new runner-owned coordinator merge_wave_worktrees. This delivers the headline-open acceptance criterion of #284: Wave-based parallelism for independent subtasks.

The core problem

Every worktree of a wave is cut off the same base (HEAD at wave start), so they cannot be merged one at a time — the first merge_subtask_worktree advances HEAD and the next trips its strict BASE_DIVERGED guard.

Design (llm-council-reviewed, conv c29d6fa9)

  • Dedicated coordinator, not a flag — zero blast radius on the shipped single-subtask path. The two share the extracted _wt_freeze_and_verify primitive (commit + per-worktree guards + pre-merge verify) but stay separate compositions. wave_base_sha is derived from the sidecar, never a caller parameter.
  • Wave-scoped divergence guard — refuses EXTERNAL HEAD movement (EXTERNAL_HEAD_MOVED) but allows the sibling divergence each in-wave squash-merge creates.
  • Merge by frozen SHA, sorted id order — one runner commit per subtask (one-commit-per-subtask contract holds).
  • All-or-nothing — any textual conflict, commit failure, or post-wave-gate failure rolls the whole branch back to base via a hard reset + clean -fd (squash leaves no MERGE_HEAD, so merge --abort is never used; MAP runtime state is excluded from the clean) and leaves every worktree intact for retry.
  • One post-wave full gate inside the atomic transaction — catches a semantic break two subtasks create together (A renames a symbol B references) that no textual merge can see.
  • Safety extras — advisory flock serializes coordinators (MERGE_IN_PROGRESS); attached/clean-target preconditions; conflicted paths attributed back to the subtasks that touched them; actual-overlap telemetry (declared-disjoint affected_files is only a scheduler hint).

What changed

  • Runner: merge_wave_worktrees + helpers (_wt_rollback, _wt_unmerged_paths, _wt_changed_files, _wt_overlap_pairs, _wt_attribute_conflict, advisory-lock helpers) + CLI dispatch; refactored merge_subtask_worktree onto the shared _wt_freeze_and_verify primitive (single-subtask behavior byte-identical — all 30 existing tests pass).
  • Skill wiring: map-efficient SKILL.md + efficient-reference.md (claude + codex variants).
  • Docs: ARCHITECTURE Phase 2 section, USAGE parallel-waves, CHANGELOG.
  • Tests: 12 wave-coordinator cases — happy path (sorted order, 2 disjoint subtasks), conflict full rollback + attribution + no MERGE_HEAD, external-head refusal, post-wave fail rollback / pass accept, preflight verify-fail abort-before-merge, no-change subtask, overlap telemetry, CLI happy + error-exit.
  • Drive-by: widened build_minimality_rollout_report -> dict[str, Any], clearing 19 pre-existing pyright errors in tests/test_minimality_report.py (the dict[str,object] __getitem__ barrier).

Testing

make check green locally: ruff + mypy + pyright 0 (src AND tests), 2922 tests passed, render check (4 generated trees byte-identical). CI on this repo is currently down (billing), so the local gate is the authority.

Deferred (issue stays open)

  • Phase 3 — context-budget hooks (statusline, threshold warnings, heartbeat).

Part of #284

…#284 Phase 2)

Wires the existing wave/DAG scheduler to per-subtask worktree isolation so a
parallel wave's independent subtasks each run in their own worktree and are
accepted atomically via a new runner-owned coordinator `merge_wave_worktrees`.

Every worktree of a wave is cut off the same base, so they can't be merged one
at a time — the first `merge_subtask_worktree` advances HEAD and the next trips
`BASE_DIVERGED`. The coordinator relaxes only that guard to a wave-scoped form
(refuses EXTERNAL HEAD movement, allows sibling divergence), derives
`wave_base_sha` from the sidecar, preflights every worktree (commit + guards +
pre-merge verify) before touching the working branch, squash-merges each
accepted worktree by frozen SHA in sorted id order (one runner commit per
subtask), then runs one post-wave full gate on the merged tree INSIDE the atomic
transaction.

All-or-nothing (council-reviewed, conv c29d6fa9): any conflict, commit failure,
or post-wave-gate failure rolls the whole branch back to base via reset --hard +
clean -fd (squash leaves no MERGE_HEAD, so merge --abort is never used; MAP
runtime state excluded from clean) and leaves every worktree intact for retry.
Safety: advisory flock serializes coordinators (MERGE_IN_PROGRESS);
attached/clean-target preconditions; conflict attribution back to the subtasks
that touched each file; actual-overlap telemetry (declared-disjoint is only a
scheduler hint, git's textual conflict stays the hard guard). The shared
`_wt_freeze_and_verify` primitive is extracted once and reused by both the
single-subtask and wave merge paths.

- runner: merge_wave_worktrees + helpers + CLI; refactor merge_subtask_worktree
  onto the shared primitive (single-subtask behavior byte-identical, 30 tests)
- skill wiring: map-efficient SKILL.md + efficient-reference.md (claude + codex)
- docs: ARCHITECTURE Phase 2 section, USAGE parallel-waves, CHANGELOG
- tests: 12 wave-coordinator cases (happy path, conflict rollback, external-head,
  post-wave fail/pass, preflight verify fail, no-change, overlap, CLI x2)
- drive-by: widen build_minimality_rollout_report -> dict[str, Any], clearing 19
  pre-existing pyright errors in tests/test_minimality_report.py (dict[str,object]
  __getitem__ barrier)

make check green: ruff/mypy/pyright 0 (src+tests), 2922 tests, render check.
@azalio azalio merged commit 8323fe5 into main Jun 27, 2026
6 checks passed
@azalio azalio deleted the feat/284-wave-worktree-parallelism branch June 27, 2026 11:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant