You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Relocatable _build for fast git worktrees (parallel coding agents): move a build instead of recompiling #6855
Coding agents increasingly work on a Phoenix app in parallel, each in its own git worktree (Claude Code --worktree and subagents with worktree isolation, Conductor, worktrunk, git-cow). On copy-on-write filesystems (APFS clonefile, btrfs/XFS reflinks) a new worktree can get _build and deps from the main checkout for free, in milliseconds. Since Elixir v1.15 Mix then recompiles the whole project anyway, because the manifest's cwd is part of the compiler cache key (elixir-lang/elixir#12462): every .beam names its source by absolute path, and __DIR__ is absolute too. elixir-lang/elixir#13241 discussed this for CI caches; --no-check-cwd (v1.19.2) skips the check but leaves the stale paths in place (stack traces, ex_doc links, __DIR__).
I measured what it takes to move a build correctly instead, on a Phoenix 1.8 app (67 modules plus a path dependency, Elixir 1.20.4, OTP 28):
Full recompile after the move: 3.8 s, the dominant cost of creating a worktree.
Rewriting the old root to the new one instead: 0.14 s, after which Mix compiles nothing except the embed_templates modules (below). The paths live in a small, known set of places:
the compile manifest: cwd and the local deps config
.beam chunks: CInf (:source), Dbgi (:file), Attr (e.g. @external_resource), LitT (compile-time literals such as __DIR__)
the .app file (compile-time config computed from __DIR__)
After that, module_info(:compile)[:source], stack traces, Tidewave and LiveDebugger all point into the new checkout. It is ~100 lines of :beam_lib.
End to end, a new worktree with its own database (PostgreSQL 18 CREATE DATABASE ... STRATEGY FILE_COPY with file_copy_method = clone) and its own mix phx.server is serving in ~2.8 s instead of ~35 s.
Ideas, in order of ambition (most of it lives in Mix, but Phoenix apps are where this hurts most):
mix compile --relocate-from OLD_ROOT, or relocation done automatically when only the cwd changed: Mix rewrites the paths above instead of recompiling. Everything it needs is already in the manifest.
Exact relocation: the compiler knows when it expands __DIR__ / __ENV__.file; recording that per module would let Mix recompile only the modules that captured the old root in a way it cannot rewrite.
Longer term: store source paths relative to the project root in .beam files (as :relative_paths already does for quoted nodes and diagnostics) and resolve them at runtime.
Phoenix side: path-independent recompile checks (phoenix_template#15), and perhaps generator defaults that avoid baking __DIR__-derived absolute paths into compile-time config.
This would make worktree-per-agent setups start in a couple of seconds instead of paying a full compile per task, and would also help CI caches and container builds that move _build between paths. Happy to work on PRs for whichever of these you'd accept.
Coding agents increasingly work on a Phoenix app in parallel, each in its own git worktree (Claude Code
--worktreeand subagents with worktree isolation, Conductor, worktrunk, git-cow). On copy-on-write filesystems (APFSclonefile, btrfs/XFS reflinks) a new worktree can get_buildanddepsfrom the main checkout for free, in milliseconds. Since Elixir v1.15 Mix then recompiles the whole project anyway, because the manifest's cwd is part of the compiler cache key (elixir-lang/elixir#12462): every .beam names its source by absolute path, and__DIR__is absolute too. elixir-lang/elixir#13241 discussed this for CI caches;--no-check-cwd(v1.19.2) skips the check but leaves the stale paths in place (stack traces, ex_doc links,__DIR__).I measured what it takes to move a build correctly instead, on a Phoenix 1.8 app (67 modules plus a path dependency, Elixir 1.20.4, OTP 28):
Full recompile after the move: 3.8 s, the dominant cost of creating a worktree.
Rewriting the old root to the new one instead: 0.14 s, after which Mix compiles nothing except the
embed_templatesmodules (below). The paths live in a small, known set of places:cwdand the local deps configCInf(:source),Dbgi(:file),Attr(e.g.@external_resource),LitT(compile-time literals such as__DIR__).appfile (compile-time config computed from__DIR__)After that,
module_info(:compile)[:source], stack traces, Tidewave and LiveDebugger all point into the new checkout. It is ~100 lines of:beam_lib.The remaining recompiles come from
embed_templates:__mix_recompile__?/0compares an md5 of the templates' absolute paths (__mix_recompile__?/0hashes absolute template paths, so a moved or cloned project recompiles everyembed_templatesmodule phoenix_template#15, with a two-line fix).End to end, a new worktree with its own database (PostgreSQL 18
CREATE DATABASE ... STRATEGY FILE_COPYwithfile_copy_method = clone) and its ownmix phx.serveris serving in ~2.8 s instead of ~35 s.Ideas, in order of ambition (most of it lives in Mix, but Phoenix apps are where this hurts most):
mix compile --relocate-from OLD_ROOT, or relocation done automatically when only the cwd changed: Mix rewrites the paths above instead of recompiling. Everything it needs is already in the manifest.__DIR__/__ENV__.file; recording that per module would let Mix recompile only the modules that captured the old root in a way it cannot rewrite.:relative_pathsalready does for quoted nodes and diagnostics) and resolve them at runtime.__DIR__-derived absolute paths into compile-time config.This would make worktree-per-agent setups start in a couple of seconds instead of paying a full compile per task, and would also help CI caches and container builds that move
_buildbetween paths. Happy to work on PRs for whichever of these you'd accept.