feat(map-efficient): parallel-wave worktree merge coordinator (Part of #284 Phase 2)#297
Merged
Merged
Conversation
…#284 Phase 2) Wires the existing wave/DAG scheduler to per-subtask worktree isolation so a parallel wave's independent subtasks each run in their own worktree and are accepted atomically via a new runner-owned coordinator `merge_wave_worktrees`. Every worktree of a wave is cut off the same base, so they can't be merged one at a time — the first `merge_subtask_worktree` advances HEAD and the next trips `BASE_DIVERGED`. The coordinator relaxes only that guard to a wave-scoped form (refuses EXTERNAL HEAD movement, allows sibling divergence), derives `wave_base_sha` from the sidecar, preflights every worktree (commit + guards + pre-merge verify) before touching the working branch, squash-merges each accepted worktree by frozen SHA in sorted id order (one runner commit per subtask), then runs one post-wave full gate on the merged tree INSIDE the atomic transaction. All-or-nothing (council-reviewed, conv c29d6fa9): any conflict, commit failure, or post-wave-gate failure rolls the whole branch back to base via reset --hard + clean -fd (squash leaves no MERGE_HEAD, so merge --abort is never used; MAP runtime state excluded from clean) and leaves every worktree intact for retry. Safety: advisory flock serializes coordinators (MERGE_IN_PROGRESS); attached/clean-target preconditions; conflict attribution back to the subtasks that touched each file; actual-overlap telemetry (declared-disjoint is only a scheduler hint, git's textual conflict stays the hard guard). The shared `_wt_freeze_and_verify` primitive is extracted once and reused by both the single-subtask and wave merge paths. - runner: merge_wave_worktrees + helpers + CLI; refactor merge_subtask_worktree onto the shared primitive (single-subtask behavior byte-identical, 30 tests) - skill wiring: map-efficient SKILL.md + efficient-reference.md (claude + codex) - docs: ARCHITECTURE Phase 2 section, USAGE parallel-waves, CHANGELOG - tests: 12 wave-coordinator cases (happy path, conflict rollback, external-head, post-wave fail/pass, preflight verify fail, no-change, overlap, CLI x2) - drive-by: widen build_minimality_rollout_report -> dict[str, Any], clearing 19 pre-existing pyright errors in tests/test_minimality_report.py (dict[str,object] __getitem__ barrier) make check green: ruff/mypy/pyright 0 (src+tests), 2922 tests, render check.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Wires the existing wave/DAG scheduler to per-subtask worktree isolation (Phase 1, #296) so a parallel wave of independent subtasks each runs in its own worktree and is accepted atomically via a new runner-owned coordinator
merge_wave_worktrees. This delivers the headline-open acceptance criterion of #284: Wave-based parallelism for independent subtasks.The core problem
Every worktree of a wave is cut off the same base (HEAD at wave start), so they cannot be merged one at a time — the first
merge_subtask_worktreeadvances HEAD and the next trips its strictBASE_DIVERGEDguard.Design (llm-council-reviewed, conv
c29d6fa9)_wt_freeze_and_verifyprimitive (commit + per-worktree guards + pre-merge verify) but stay separate compositions.wave_base_shais derived from the sidecar, never a caller parameter.EXTERNAL_HEAD_MOVED) but allows the sibling divergence each in-wave squash-merge creates.clean -fd(squash leaves noMERGE_HEAD, somerge --abortis never used; MAP runtime state is excluded from the clean) and leaves every worktree intact for retry.flockserializes coordinators (MERGE_IN_PROGRESS); attached/clean-target preconditions; conflicted paths attributed back to the subtasks that touched them; actual-overlap telemetry (declared-disjointaffected_filesis only a scheduler hint).What changed
merge_wave_worktrees+ helpers (_wt_rollback,_wt_unmerged_paths,_wt_changed_files,_wt_overlap_pairs,_wt_attribute_conflict, advisory-lock helpers) + CLI dispatch; refactoredmerge_subtask_worktreeonto the shared_wt_freeze_and_verifyprimitive (single-subtask behavior byte-identical — all 30 existing tests pass).map-efficientSKILL.md +efficient-reference.md(claude + codex variants).MERGE_HEAD, external-head refusal, post-wave fail rollback / pass accept, preflight verify-fail abort-before-merge, no-change subtask, overlap telemetry, CLI happy + error-exit.build_minimality_rollout_report -> dict[str, Any], clearing 19 pre-existing pyright errors intests/test_minimality_report.py(thedict[str,object]__getitem__barrier).Testing
make checkgreen locally: ruff + mypy + pyright 0 (src AND tests), 2922 tests passed, render check (4 generated trees byte-identical). CI on this repo is currently down (billing), so the local gate is the authority.Deferred (issue stays open)
Part of #284