Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Fixed

- Git-subpath `GitCache` checkouts pin `core.autocrlf=false` so host autocrlf settings do not rewrite LF-committed package content; older unpinned cache shards rematerialize on the next install. (closes #2971) (#2982)
- GitLab `path:` dependencies now preserve the selected SSH transport, username, and port instead of silently using HTTPS; REST fallback requires an executed same-origin HTTPS attempt admitted by the transport policy. (#2938)

## [0.30.0] - 2026-09-07
Expand Down
2 changes: 1 addition & 1 deletion docs/src/content/docs/reference/lockfile-spec.md
Original file line number Diff line number Diff line change
Expand Up @@ -217,7 +217,7 @@ Each item in `dependencies` describes one resolved package.
| `resolved_url` | string | registry only | Fully-qualified download URL used to re-fetch registry archives. |
| `resolved_hash` | string | registry only | SHA-256 digest of the registry archive bytes, verified on every install. |
| `local_path` | string | no | Original path from `apm.yml` for local deps, relative to project root. |
| `content_hash` | string | no | SHA-256 of the materialized package tree, computed from sorted relative paths and raw file bytes. For remote dependencies it verifies that downloaded or cached content still matches the lock; for local path dependencies it detects source-tree changes. |
| `content_hash` | string | no | SHA-256 of the materialized package tree, computed from sorted relative paths and raw file bytes. GitCache-backed git-subpath checkouts pin `core.autocrlf=false` so LF-committed content is not rewritten as CRLF on checkout; this pin does not override `.gitattributes` or `core.eol`. Bytes that are committed as CRLF stay CRLF. For remote dependencies it verifies that downloaded or cached content still matches the lock; for local path dependencies it detects source-tree changes. |
| `is_dev` | bool | no | `true` when the dep was declared under `devDependencies`. |
| `discovered_via` | string | no | Marketplace name that surfaced this package (provenance). |
| `marketplace_plugin_name` | string | no | Plugin name as listed in that marketplace. |
Expand Down
4 changes: 2 additions & 2 deletions docs/src/content/docs/troubleshooting/common-errors.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,9 +85,9 @@ indicate a supply-chain attack. Use 'apm install --update' to accept
new content and update the lockfile.
```

Cause: the bytes APM downloaded for a locked dependency do not match the `content_hash` recorded in `apm.lock.yaml`. Usually a force-pushed tag, a mutated branch, or a tampered mirror.
Cause: the bytes APM downloaded for a locked dependency do not match the `content_hash` recorded in `apm.lock.yaml`. Usually a force-pushed tag, a mutated branch, or a tampered mirror. A Windows host with Git's default `core.autocrlf=true` used to record CRLF hashes for git-subpath packages. Current APM pins `core.autocrlf=false` on GitCache checkouts and rematerializes older unpinned cache shards, but `--frozen` cannot rewrite a lockfile that already stored that CRLF hash.

Fix: investigate the upstream package before accepting. If the change is legitimate, run `apm install --update` to re-pin and commit the new hash.
Fix: investigate the upstream package before accepting. If the change is legitimate, run `apm install --update` to re-pin and commit the new hash. `--update` can advance mutable refs, so review the SHA and `content_hash` diff before committing. If the lockfile was generated by a pre-fix Windows CLI, rematerialize the affected package from a fresh consumer workspace (or delete the cached/installed copy) and regenerate with `apm install --update` or `apm lock --update`. Bare `apm lock` does not rewrite an existing CRLF hash.

See also: [Install failures](../install-failures/), [Lockfile specification](../../reference/lockfile-spec/)

Expand Down
94 changes: 89 additions & 5 deletions src/apm_cli/cache/git_cache.py
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,13 @@ def _safe_git_args() -> list[str]:
malicious upstream might ship, so clone and checkout stay inert.
- ``submodule.recurse=false`` prevents any subcommand from
recursing into attacker-controlled submodule URLs.
- ``core.autocrlf=false`` disables host autocrlf conversion so
LF-committed blobs are not rewritten as CRLF on checkout.
``-c`` outranks host system / global config and
``GIT_CONFIG_KEY_n`` snapshots from ``git_network_env``, which
otherwise win over a repo-local pin (apm#2971). This pin does
not override ``core.eol`` or ``.gitattributes`` ``eol=crlf`` /
``text=auto`` requests.

These flags are scoped per-invocation via ``-c`` and never mutate
the user's gitconfig. The cache layer is the single source of
Expand All @@ -79,9 +86,44 @@ def _safe_git_args() -> list[str]:
*git_no_hooks_args(),
"-c",
"submodule.recurse=false",
"-c",
"core.autocrlf=false",
]


def _checkout_pins_autocrlf_false(checkout_dir: Path) -> bool:
"""Return whether the checkout's local gitconfig pins ``core.autocrlf=false``.

Pre-fix shards materialized under host ``core.autocrlf=true`` omit this
pin and may contain CRLF working-tree bytes. Cache hits rematerialize
those shards so existing Windows caches heal without ``apm cache clean``.
"""
config = checkout_dir / ".git" / "config"
if not config.is_file():
return False
try:
text = config.read_text(encoding="utf-8")
except (OSError, UnicodeError):
return False
section = None
for raw in text.splitlines():
line = raw.split(";", 1)[0].split("#", 1)[0].strip()
if not line:
continue
if line.startswith("[") and line.endswith("]"):
inner = line[1:-1].strip()
section = inner.split(" ", 1)[0].strip().lower()
continue
if section != "core" or "=" not in line:
continue
key, _, value = line.partition("=")
if key.strip().lower() != "autocrlf":
continue
normalized = value.strip().strip("\"'").lower()
return normalized in {"false", "0", "no", "off"}
return False


# Partial bare-cache flavor suffix (perf #1433 follow-up).
# When a caller requests sparse_paths, we use a separate bare keyed at
# ``<shard>__p`` cloned with ``--filter=blob:none``. The partial bare
Expand Down Expand Up @@ -201,13 +243,14 @@ def get_checkout(

# Cache hit path (skip if refresh requested)
if not self._refresh and checkout_dir.is_dir():
if verify_checkout_sha(checkout_dir, sha):
sha_ok = verify_checkout_sha(checkout_dir, sha)
if sha_ok and _checkout_pins_autocrlf_false(checkout_dir):
_log.debug("Cache HIT: %s @ %s [%s]", _sanitize_url(url), sha[:12], variant)
with shard_lock(checkout_dir):
return self._record_checkout_access(
self._finalize_sparse_checkout(url, checkout_dir, sparse_paths, env=env)
)
else:
elif not sha_ok:
# Integrity failure -- evict
_log.warning(
"[!] Evicting corrupt cache entry: %s @ %s [%s]",
Expand All @@ -216,6 +259,9 @@ def get_checkout(
variant,
)
self._evict_checkout(checkout_dir)
# SHA-valid unpinned trees stay until ``_create_checkout`` holds
# ``shard_lock`` and emits the rematerialize log. A concurrent
# consumer may still be reading the old checkout.

# Cache miss: ensure we have the bare repo, then create checkout.
# Sparse callers use a partial bare (blob:none) + promisor consumer
Expand Down Expand Up @@ -614,7 +660,8 @@ def _create_checkout(
# this shard while we were waiting. Verify integrity to
# rule out a poisoned half-write (atomic_land guards
# against that, but we re-check defensively).
if final_dir.is_dir() and verify_checkout_sha(final_dir, sha):
existing_ok = final_dir.is_dir() and verify_checkout_sha(final_dir, sha)
if existing_ok and _checkout_pins_autocrlf_false(final_dir):
_log.debug(
"Write-dedup HIT under lock: %s @ %s [%s]",
_sanitize_url(url),
Expand All @@ -624,6 +671,20 @@ def _create_checkout(
return self._record_checkout_access(
self._finalize_sparse_checkout(url, final_dir, sparse_paths, env=env)
)
if existing_ok:
_log.info(
"[*] Rematerializing git checkout missing core.autocrlf=false pin: "
"%s @ %s [%s]",
_sanitize_url(url),
Comment thread
sergio-sisternes-epam marked this conversation as resolved.
sha[:12],
variant,
)
self._evict_checkout(final_dir)
if final_dir.exists():
raise RuntimeError(
"Failed to rematerialize unpinned git checkout "
f"for {_sanitize_url(url)} @ {sha[:12]}"
)

staged = stage_path(final_dir)
ensure_path_within(staged, self._checkouts_root)
Expand Down Expand Up @@ -670,6 +731,27 @@ def _create_checkout(
stdin=subprocess.DEVNULL,
check=True,
)
# Persist the pin so cache hits can recognize post-fix shards.
# Checkout itself still needs ``-c core.autocrlf=false`` from
# ``_safe_git_args`` because env-frozen host config outranks
# this local value.
subprocess.run(
[
git_exe,
*_safe_git_args(),
"-C",
str(staged),
"config",
"core.autocrlf",
"false",
],
capture_output=True,
text=True,
timeout=10,
env=subprocess_env,
stdin=subprocess.DEVNULL,
check=True,
)
if promisor_url:
# Point origin at the real upstream (clone set it to the
# local bare). Single config call; the other two promisor
Expand Down Expand Up @@ -755,8 +837,10 @@ def _create_checkout(
if not atomic_land(staged, final_dir, lock):
# Another process landed first between our re-probe and
# the rename (only possible if our lock dropped, which
# it didn't); verify integrity defensively.
if not verify_checkout_sha(final_dir, sha):
# it didn't); verify integrity and the autocrlf pin.
if not (
verify_checkout_sha(final_dir, sha) and _checkout_pins_autocrlf_false(final_dir)
):
self._evict_checkout(final_dir)
raise RuntimeError(
f"Race condition: concurrent checkout failed integrity "
Expand Down
15 changes: 12 additions & 3 deletions tests/integration/test_git_cache_hermetic.py
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,10 @@ def test_cache_hit_returns_existing_checkout(self, cache: GitCache) -> None:
sha = "a" * 40
checkout_dir = cache._checkouts_root / cache_shard_key(url) / sha / "full"
checkout_dir.mkdir(parents=True)
(checkout_dir / ".git").mkdir()
(checkout_dir / ".git" / "config").write_text(
"[core]\n\tautocrlf = false\n", encoding="ascii"
)

with (
patch.object(cache, "_resolve_sha", return_value=sha),
Expand Down Expand Up @@ -494,6 +498,8 @@ def test_write_dedup_hit_under_lock_returns_existing_checkout(self, cache: GitCa
shard_key = cache_shard_key(url)
final_dir = cache._checkouts_root / shard_key / ("a" * 40) / "full"
final_dir.mkdir(parents=True)
(final_dir / ".git").mkdir()
(final_dir / ".git" / "config").write_text("[core]\n\tautocrlf = false\n", encoding="ascii")

with (
patch("apm_cli.cache.git_cache.shard_lock", return_value=nullcontext()),
Expand Down Expand Up @@ -533,7 +539,7 @@ def _land(staged: Path, final: Path, _lock: object) -> bool:
result = cache._create_checkout(url, shard_key, "b" * 40)

assert result == final_dir
assert mock_run.call_count == 2
assert mock_run.call_count == 3

def test_clone_failure_cleans_staged_checkout(self, cache: GitCache) -> None:
url = "https://example.com/repo.git"
Expand Down Expand Up @@ -562,10 +568,11 @@ def test_checkout_failure_cleans_staged_checkout(self, cache: GitCache) -> None:
(cache._db_root / shard_key).mkdir(parents=True)

clone_result = _proc()
config_result = _proc()
checkout_error = subprocess.CalledProcessError(1, "git", stderr="checkout failed")
with (
patch("apm_cli.cache.git_cache.shard_lock", return_value=nullcontext()),
patch("subprocess.run", side_effect=[clone_result, checkout_error]),
patch("subprocess.run", side_effect=[clone_result, config_result, checkout_error]),
patch("apm_cli.utils.git_env.get_git_executable", return_value="git"),
patch("apm_cli.utils.git_env.git_subprocess_env", return_value={}),
patch("apm_cli.cache.git_cache.os.chmod"),
Expand All @@ -581,6 +588,8 @@ def test_atomic_land_false_accepts_valid_winner(self, cache: GitCache) -> None:
shard_key = cache_shard_key(url)
final_dir = cache._checkouts_root / shard_key / ("e" * 40) / "full"
final_dir.mkdir(parents=True)
(final_dir / ".git").mkdir()
(final_dir / ".git" / "config").write_text("[core]\n\tautocrlf = false\n", encoding="ascii")
(cache._db_root / shard_key).mkdir(parents=True)

verify_results = [False, True]
Expand All @@ -596,7 +605,7 @@ def test_atomic_land_false_accepts_valid_winner(self, cache: GitCache) -> None:
result = cache._create_checkout(url, shard_key, "e" * 40)

assert result == final_dir
assert mock_run.call_count == 2
assert mock_run.call_count == 3

def test_atomic_land_false_with_invalid_winner_evicts_and_raises(self, cache: GitCache) -> None:
url = "https://example.com/repo.git"
Expand Down
6 changes: 6 additions & 0 deletions tests/unit/cache/test_git_cache.py
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,9 @@ def test_cache_hit_with_integrity_pass(self, mock_run: MagicMock, tmp_path: Path
checkout_dir = tmp_path / "git" / "checkouts_v1" / real_shard / sha / "full"
checkout_dir.mkdir(parents=True)
(checkout_dir / ".git").mkdir()
(checkout_dir / ".git" / "config").write_text(
"[core]\n\tautocrlf = false\n", encoding="ascii"
)

# Mock git rev-parse HEAD to return the expected SHA
mock_run.return_value = MagicMock(
Expand Down Expand Up @@ -325,6 +328,7 @@ def test_short_circuits_when_final_exists_under_lock(
final_dir = tmp_path / "git" / "checkouts_v1" / shard / sha / "full"
final_dir.mkdir(parents=True)
(final_dir / ".git").mkdir()
(final_dir / ".git" / "config").write_text("[core]\n\tautocrlf = false\n", encoding="ascii")

with (
patch("subprocess.run") as mock_run,
Expand Down Expand Up @@ -388,6 +392,8 @@ def test_short_circuits_on_integrity_pass_only(self, tmp_path: Path) -> None:
# Populate final_dir BUT integrity will report failure.
final_dir = tmp_path / "git" / "checkouts_v1" / shard / sha / "full"
final_dir.mkdir(parents=True)
(final_dir / ".git").mkdir()
(final_dir / ".git" / "config").write_text("[core]\n\tautocrlf = false\n", encoding="ascii")
(tmp_path / "git" / "db_v1" / shard).mkdir(parents=True)

def _populate(*args, **kwargs):
Expand Down
Loading