Skip to content

Relocatable _build for fast git worktrees (parallel coding agents): move a build instead of recompiling #6855

Description

@onnimonni

Coding agents increasingly work on a Phoenix app in parallel, each in its own git worktree (Claude Code --worktree and subagents with worktree isolation, Conductor, worktrunk, git-cow). On copy-on-write filesystems (APFS clonefile, btrfs/XFS reflinks) a new worktree can get _build and deps from the main checkout for free, in milliseconds. Since Elixir v1.15 Mix then recompiles the whole project anyway, because the manifest's cwd is part of the compiler cache key (elixir-lang/elixir#12462): every .beam names its source by absolute path, and __DIR__ is absolute too. elixir-lang/elixir#13241 discussed this for CI caches; --no-check-cwd (v1.19.2) skips the check but leaves the stale paths in place (stack traces, ex_doc links, __DIR__).

I measured what it takes to move a build correctly instead, on a Phoenix 1.8 app (67 modules plus a path dependency, Elixir 1.20.4, OTP 28):

  • Full recompile after the move: 3.8 s, the dominant cost of creating a worktree.

  • Rewriting the old root to the new one instead: 0.14 s, after which Mix compiles nothing except the embed_templates modules (below). The paths live in a small, known set of places:

    • the compile manifest: cwd and the local deps config
    • .beam chunks: CInf (:source), Dbgi (:file), Attr (e.g. @external_resource), LitT (compile-time literals such as __DIR__)
    • the .app file (compile-time config computed from __DIR__)

    After that, module_info(:compile)[:source], stack traces, Tidewave and LiveDebugger all point into the new checkout. It is ~100 lines of :beam_lib.

  • The remaining recompiles come from embed_templates: __mix_recompile__?/0 compares an md5 of the templates' absolute paths (__mix_recompile__?/0 hashes absolute template paths, so a moved or cloned project recompiles every embed_templates module phoenix_template#15, with a two-line fix).

End to end, a new worktree with its own database (PostgreSQL 18 CREATE DATABASE ... STRATEGY FILE_COPY with file_copy_method = clone) and its own mix phx.server is serving in ~2.8 s instead of ~35 s.

Ideas, in order of ambition (most of it lives in Mix, but Phoenix apps are where this hurts most):

  1. mix compile --relocate-from OLD_ROOT, or relocation done automatically when only the cwd changed: Mix rewrites the paths above instead of recompiling. Everything it needs is already in the manifest.
  2. Exact relocation: the compiler knows when it expands __DIR__ / __ENV__.file; recording that per module would let Mix recompile only the modules that captured the old root in a way it cannot rewrite.
  3. Longer term: store source paths relative to the project root in .beam files (as :relative_paths already does for quoted nodes and diagnostics) and resolve them at runtime.
  4. Phoenix side: path-independent recompile checks (phoenix_template#15), and perhaps generator defaults that avoid baking __DIR__-derived absolute paths into compile-time config.

This would make worktree-per-agent setups start in a couple of seconds instead of paying a full compile per task, and would also help CI caches and container builds that move _build between paths. Happy to work on PRs for whichever of these you'd accept.

Activity

  1. SteffenDE commented on Oct 1, 2026

    @SteffenDE
    Member

    I think there's not much to do on the Phoenix side after phoenixframework/phoenix_template#16?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions