fix: authenticate private subdirectory cache (closes #2714) - #2722
fix: authenticate private subdirectory cache (closes #2714)#2722Daniel Meppiel (danielmeppiel) wants to merge 8 commits into
Conversation
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Copilot review overview
Review tier: Lite
Findings: 2
New issues introduced by this change (2)
| Severity | Finding |
|---|---|
tests/unit/cache/test_git_cache_sparse.py — This test no longer exercises URL credential redaction: the input URL is already redacted (no… |
|
src/apm_cli/deps/github_downloader.py — The env produced by AuthResolver.try_with_fallback is passed straight into GitCache.get_checkout… |
What changed in this PR
This PR fixes a credential-routing gap where persistent Git cache population for private github.com subdirectory dependencies could run in an anonymous-only git environment, fail authentication, and be discarded (bypassing the persistent cache even though the later authenticated download path succeeds). The change routes persistent-cache clone/checkout work through AuthResolver’s anonymous-first + path-scoped fallback so auth failures trigger the existing credential resolution chain.
Changes:
- Add a
github_downloaderhelper to run persistent-cacheget_checkout()throughAuthResolver.try_with_fallback()for publicgithub.comanonymous-first flows (including subdirectory sparse cache). - Improve auth-failure classification to read wrapped exception stderr/stdout and tighten partial-clone fallback warnings to only mention filter-v2 when stderr supports that diagnosis.
- Add unit + architecture boundary tests and update docs to reflect persistent-cache participation in the path-scoped fallback behavior.
| File | Description |
|---|---|
| tests/unit/core/test_public_github_anonymous_first.py | Adds unit coverage for wrapped-git stderr auth classification and persistent-cache retry behavior for private subdirectory installs. |
| tests/unit/cache/test_git_cache_sparse.py | Adds coverage for partial-clone fallback warning wording and credential redaction. |
| tests/integration/test_architecture_authorities.py | Adds an integration guard test ensuring persistent-cache paths keep routing through AuthResolver. |
| src/apm_cli/deps/github_downloader.py | Introduces _persistent_cache_checkout() and uses it for both subdirectory and whole-repo persistent-cache population. |
| src/apm_cli/core/auth.py | Extends is_public_github_auth_failure() to walk exception chains so wrapped git stderr still unlocks fallback. |
| src/apm_cli/cache/git_cache.py | Adds _partial_clone_fallback_warning() and uses it in partial-clone retry warnings. |
| scripts/lint-architecture-boundaries.sh | Extends the boundary lint to ensure persistent-cache auth routing stays owned by AuthResolver. |
| packages/apm-guide/.apm/skills/apm-usage/authentication.md | Documents that persistent Git cache population participates in path-scoped fallback and never persists credentials. |
| docs/src/content/docs/getting-started/authentication.md | Updates end-user docs to include persistent-cache population in the per-repo cached fallback behavior. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| warning = _partial_clone_fallback_warning( | ||
| "https://user:secret@github.com/acme/private", | ||
| failure, | ||
| ) | ||
|
|
||
| assert "filter v2" not in warning | ||
| assert "secret" not in warning |
| def _checkout(_token: str | None, env: dict[str, str]) -> Path: | ||
| return cache.get_checkout( | ||
| repository_url, | ||
| ref, | ||
| locked_sha=locked_sha, | ||
| env=env, | ||
| sparse_paths=sparse_paths, | ||
| ) |
APM Review Panel:
|
| Persona | B | R | N | Takeaway |
|---|---|---|---|---|
| Python Architect | 1 | 1 | 0 | AuthResolver remains canonical, but a stale AC11 assertion fails and AC20 guards only the helper body. |
| CLI Logging Expert | 0 | 1 | 0 | Redaction and ASCII output are sound, but the partial-clone warning fires before the final outcome. |
| DevX UX Expert | 1 | 1 | 0 | The cache fallback cannot authenticate GitHub subprocesses and successful private installs emit a misleading warning. |
| Supply Chain Security Expert | 1 | 0 | 0 | Private GitHub cache retries resolve credentials but never provide them to Git. |
| OSS Growth Hacker | 0 | 1 | 0 | The fix removes private-package friction; add a user-shaped changelog entry. |
| Auth Expert | 1 | 2 | 0 | The retry supplies neither Git-consumable auth nor a sanitized subprocess environment. |
| Doc Writer | 0 | 0 | 0 | Documentation is concise, discoverable, and accurate. |
| Test Coverage Expert | 1 | 3 | 0 | Whole-repo E2E fails; sparse and credential-persistence promises remain below fixture-backed tier. |
| Performance Expert | 0 | 1 | 0 | Cold private sparse misses perform a redundant anonymous full-clone negotiation. |
B = blocking-severity findings, R = recommended, N = nits.
Counts are signal strength, not gates. The maintainer ships.
Top 5 follow-ups
- [Test Coverage Expert] (blocking-severity) Make the failing private GitHub lifecycle E2E pass through the real authenticated retry path -- the current test proves the promised lifecycle does not complete with a sanitized environment.
- [Auth Expert] (blocking-severity) Consume the resolved token through
GitAuthEnvBuilder.subprocess_env_dictfor sparse and whole-repository cache subprocesses -- this supplies Git-consumable authorization while stripping ambient Git variables. - [Python Architect] (blocking-severity) Replace the stale AC11 assertion and strengthen AC20 against caller bypasses -- both cache paths must invoke the canonical helper with a clean
dep_ref.to_github_url(). - [Test Coverage Expert] Add fixture-backed sparse, whole-repository, and durable-cache authentication coverage -- prove real authorization and credential-free URLs, remotes, and keys.
- [CLI Logging Expert] Emit clone-degradation warnings only after the final successful fallback outcome -- an expected anonymous miss may later authenticate with partial clone.
Architecture
classDiagram
class GitHubPackageDownloader {
+download_package()
+download_subdirectory_package()
-_persistent_cache_checkout()
}
class AuthResolver {
<<Strategy>>
+try_with_fallback()
+is_public_github_auth_failure()
}
class GitCache {
<<IOBoundary>>
+get_checkout()
}
class DependencyReference
GitHubPackageDownloader *-- AuthResolver : delegates auth
GitHubPackageDownloader o-- GitCache : uses cache
GitHubPackageDownloader ..> DependencyReference : reads clean identity
flowchart TD
A[download package] --> B[_persistent_cache_checkout]
B --> C{public github.com and secure}
C -->|yes| D[AuthResolver.try_with_fallback]
C -->|no| E[canonical cache Git environment]
D --> F[anonymous sanitized attempt]
F -->|auth failure| G[path-scoped credential resolution]
G --> H[authenticated sanitized Git environment]
E --> I[GitCache.get_checkout]
H --> I
I --> J[credential-free URL, remote, and cache key]
Recommendation
Revise the cache callback to use the canonical sanitized Git auth environment, then rerun the failing lifecycle and architecture tests. After those promises hold, fold the warning timing, fixture-backed persistence coverage, redaction test, performance classification, and changelog follow-up.
Full per-persona findings
Python Architect
- [blocking] Update the stale AC11 cache identity assertion at
tests/integration/test_architecture_authorities.py:3779.
The test still requires a direct call that this PR intentionally replaced with owner-backed routing. - [recommended] Guard persistent-cache call sites, not only the helper body at
scripts/lint-architecture-boundaries.sh:1337.
AC20 must reject either downloader path bypassing_persistent_cache_checkout().
CLI Logging Expert
- [recommended] Emit the partial-clone warning only after the full-clone fallback succeeds at
src/apm_cli/cache/git_cache.py:418.
An expected anonymous auth challenge can later succeed as a partial clone, making the early degradation warning false.
DevX UX Expert
- [blocking] Pass the resolved GitHub credential through a channel Git consumes at
src/apm_cli/deps/github_downloader.py:348.
The callback ignores_token; stock Git does not consumeGIT_TOKEN. - [recommended] Keep expected anonymous auth challenges off the successful install path.
Supply Chain Security Expert
- [blocking] Authenticated cache retry discards the resolved credential at
src/apm_cli/deps/github_downloader.py:348.
Use the canonical Git authorization-header helper, removeGIT_TOKEN, and retain a clean repository URL.
OSS Growth Hacker
- [recommended] Add this private-repository install fix to the Unreleased changelog.
Auth Expert
- [blocking] Private GitHub cache retries cannot authenticate and receive an unsanitized environment at
src/apm_cli/deps/github_downloader.py:348.
Sanitize withGitAuthEnvBuilder.subprocess_env_dictand install auth through the canonical header helper. - [recommended] Update the stale cache-identity architecture assertion.
- [recommended] Add integration coverage proving authenticated cache population for both routes.
Doc Writer
No findings.
Test Coverage Expert
- [blocking] Whole-repository private cache authentication fails its existing E2E contract.
test_private_github_fallback_normalizes_locale_and_completes_lifecyclefailed on this head. - [recommended] Sparse-cache retry evidence stops at a mocked cache boundary.
- [recommended] Private persistent-cache behavior lacks fixture-backed sparse and durability coverage.
- [recommended] The behavior-change PR body has no Scenario Evidence table.
Performance Expert
- [recommended] Do not retry an anonymous partial-clone auth failure as an anonymous full clone at
src/apm_cli/cache/git_cache.py:415.
Propagate auth-shaped failures immediately so AuthResolver performs the credentialed attempt.
This panel is advisory. It does not block merge. Re-apply the panel-review label after addressing feedback to re-run.
Route persistent cache credentials through AuthResolver-owned header construction, sanitize Git subprocess environments, avoid redundant auth retries, and strengthen behavioral and architecture guards. Addresses the PR 2722 review-panel and Copilot follow-ups. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
apm-spec-waiver: Restores existing private Git auth and cache behavior; no normative extension. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The installed-CLI lifecycle now fails if the expected anonymous auth challenge is mislabeled as a partial-clone degradation. Addresses the final test-coverage panel follow-up. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Treat resolver-provided environments as complete subprocess bases so removed ambient Git authorization cannot reappear. Adds callback and builder regression coverage and addresses the terminal security-panel finding. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Capture the completed full-clone fallback diagnostic so warning removal or wording drift fails deterministically. Addresses the final test-coverage panel follow-up. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Pass the complete downloader environment into AuthResolver and sanitize only each callback attempt, preserving PyInstaller-restored library paths. Addresses the terminal auth-panel finding. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Give the credential-persistence guarantee a direct active-voice sentence in both authentication guides. Addresses the terminal documentation-panel follow-up. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
APM Review Panel:
|
| Persona | B | R | N | Takeaway |
|---|---|---|---|---|
| Python Architect | 0 | 0 | 1 | Canonical AuthResolver ownership and dual guardrails cover the persistent cache path. |
| CLI Logging Expert | 0 | 0 | 0 | The fallback warning is concise, credential-safe, and emitted only after success. |
| DevX UX Expert | 0 | 0 | 0 | Private GitHub cache fallback is automatic, noninteractive, predictable, and documented. |
| Supply Chain Security Expert | 0 | 0 | 0 | Credential ownership, containment, environment sanitization, and cache identity are preserved. |
| OSS Growth Hacker | 0 | 0 | 0 | Docs, body, and changelog frame a verified reliability and security fix. |
| Auth Expert | 0 | 0 | 0 | Scoped fallback, one-pass frozen env sanitization, and credential-free persistence are correct. |
| Doc Writer | 0 | 0 | 0 | Final documentation and PR body are accurate, aligned, and discoverable. |
| Test Coverage Expert | 0 | 0 | 0 | Exact-head root and sparse cache regression traps pass. |
| Performance Expert | 0 | 0 | 0 | Redundant retries are removed while SHA and cache invariants remain intact. |
B = blocking-severity findings, R = recommended, N = nits.
Counts are signal strength, not gates. The maintainer ships.
Architecture
classDiagram
class GitHubPackageDownloader {
-_persistent_cache_checkout()
}
class AuthResolver {
<<Strategy>>
+try_with_fallback()
+build_public_github_authenticated_git_env()
}
class GitAuthEnvBuilder {
<<Adapter>>
+subprocess_env_dict()
}
class GitCache {
+get_checkout()
}
GitHubPackageDownloader --> AuthResolver : delegates credential decisions
GitHubPackageDownloader --> GitAuthEnvBuilder : sanitizes each attempt
GitHubPackageDownloader --> GitCache : uses clean URL and environment
flowchart TD
A[Persistent cache checkout] --> B[Anonymous sanitized attempt]
B --> C{Result}
C -->|success| D[Credential-free cache]
C -->|auth failure| E[AuthResolver path-scoped credential]
E --> F[GitHub Basic header environment]
F --> D
C -->|filter unsupported| G[Full bare clone fallback]
G --> H[Sanitized warning after success]
H --> D
Recommendation
Ship this exact head as-is. Specialist consensus, exact-head regressions, mutation-break evidence, and fully green CI leave no unresolved action.
Folded in this run
- (panel) Routed resolved GitHub credentials through AuthResolver-owned Basic header construction -- resolved in
dc6ee60f. - (panel) Sanitized cache callback environments and prevented ambient Git authorization rehydration -- resolved in
dc6ee60fandbcfc5c8d. - (panel) Preserved PyInstaller-restored library paths by sanitizing each callback attempt once -- resolved in
7b78aaa2. - (panel) Propagated auth failures without a redundant anonymous full clone or false warning -- resolved in
dc6ee60f. - (panel) Updated AC11 and strengthened AC20 against helper and caller bypasses -- resolved in
dc6ee60f. - (panel) Added fixture-backed root and sparse private-cache lifecycle coverage -- resolved in
dc6ee60f. - (panel) Added positive and negative fallback-warning output contracts -- resolved in
ea5e1daaanded44c9c1. - (panel) Added changelog, scenario evidence, and direct credential-free storage wording -- resolved in
dc6ee60f,bcfc5c8d, andfd6347b3. - (copilot) Replaced the tautological URL-redaction input with a credential-bearing fake URL -- resolved in
dc6ee60f. - (copilot) Routed cache callback environments through GitAuthEnvBuilder sanitization -- resolved in
dc6ee60f.
Copilot signals reviewed
tests/unit/cache/test_git_cache_sparse.py:61-- LEGIT: the original URL made the redaction assertion tautological; the replacement proves username/password sanitization (resolved indc6ee60f).src/apm_cli/deps/github_downloader.py:355-- LEGIT: cache subprocesses must strip ambient repository state on every attempt (resolved indc6ee60f).
Regression-trap evidence (mutation-break gate)
test_private_github_subdirectory_cache_retries_with_scoped_credential-- removed AuthResolver-owned header routing; test failed; guard restored.test_partial_clone_auth_failure_does_not_retry_full_clone-- removed filter classification; test failed; guard restored.test_partial_clone_warning_redacts_url_credentials-- removed URL sanitization; test failed; guard restored.test_public_github_auth_owner_guard_rejects_persistent_cache_bypass-- removed AC20 caller checks; test failed; guard restored.test_private_github_fallback_normalizes_locale_and_completes_lifecycle-- emitted the fallback warning before retry; test failed; guard restored.test_merges_auth_env_over_sanitized_base-- restored ambient authorization merging; test failed; guard restored.test_private_github_subdirectory_cache_retries_with_scoped_credential-- restored double sanitization under frozen mode; test failed; guard restored.test_partial_clone_fallback_to_full_on_server_rejection-- removed the completed fallback warning; test failed; guard restored.
Lint contract
uv run --extra dev ruff check src/ tests/ and uv run --extra dev ruff format --check src/ tests/ both passed. Pylint R0801, auth-signal lint, architecture boundary lint, and static guards also passed.
CI
All checks passed on fd6347b3f58926dd3323817a4eeb9618c4684a4e, including Lint, both test shards, Lifecycle Smoke, Architecture Ratchets, Windows Compatibility, CodeQL, spec conformance, platform probes, and merge gate (after 2 CI recovery iterations).
Mergeability status
| PR | head SHA | CEO stance | iters | folds | defers | Copilot rounds | CI | mergeable | mergeStateStatus | notes |
|---|---|---|---|---|---|---|---|---|---|---|
| #2722 | fd6347b |
ship_now | 4 | 10 | 0 | 2 | green | MERGEABLE | BLOCKED | pending required review |
Convergence
4 outer iterations; 2 Copilot rounds. Final panel stance: ship_now.
Ready for maintainer review.
Full per-persona findings
Python Architect
- [nit] Pattern inventory only; no change requested. Strategy, Chain of Responsibility, and Adapter roles remain appropriately small.
CLI Logging Expert
No findings.
DevX UX Expert
No findings.
Supply Chain Security Expert
No findings.
OSS Growth Hacker
No findings.
Auth Expert
No findings.
Doc Writer
No findings.
Test Coverage Expert
No findings.
Performance Expert
No findings.
This panel is advisory. It does not block merge. Re-apply the panel-review label after addressing feedback to re-run.

fix(auth): authenticate private GitHub persistent cache retries
TL;DR
Private
github.compackage installs now populate the persistent Git cachethrough
AuthResolver's anonymous-first, repository-scoped fallback. Thecredential reaches Git through a process-scoped Basic authorization header,
while argv, repository URLs, stored remotes, and cache keys remain credential-free.
Auth failures also skip a redundant anonymous full-clone retry and avoid a
misleading partial-clone warning.
Important
This closes #2714 and preserves
AuthResolveras the single owner of host and credential resolution.apm-spec-waiver: Restores existing private Git auth and cache behavior; no normative extension.
Problem (WHY)
that stock Git could not consume, so the persistent cache stayed empty.
allowing ambient values such as
GIT_DIRto bias cache operations.even when the failure was an expected authentication challenge.
The repair follows the repository rule that durable credential decisions have one
owner. It also applies the PROSE principle that
"Grounding outputs in deterministic tool execution transforms probabilistic generation into verifiable action."
The fixture-backed lifecycle and mutation tests are the deterministic evidence.
Approach (WHAT)
AuthResolverwith GitHub Git-header construction and route both cache consumers through it..apm/instructions/architecture.instructions.mdsrc/apm_cli/deps/git_auth_env.pysrc/apm_cli/cache/git_cache.pytests/andscripts/lint-architecture-boundaries.shImplementation (HOW)
src/apm_cli/core/auth.pysrc/apm_cli/deps/github_downloader.pysrc/apm_cli/deps/git_auth_env.pysrc/apm_cli/utils/git_env.pysrc/apm_cli/cache/git_cache.pyscripts/lint-architecture-boundaries.shtests/unit/core/test_public_github_anonymous_first.pytests/unit/cache/test_git_cache_sparse.pytests/unit/deps/test_git_auth_env.pytests/integration/test_public_github_anonymous_lifecycle_e2e.pytests/integration/test_architecture_authorities.pydocs/src/content/docs/getting-started/authentication.mdpackages/apm-guide/.apm/skills/apm-usage/authentication.mdCHANGELOG.mdDiagrams
Legend: follow the left-to-right path from a clean dependency identity through anonymous-first resolution to a credential-free persistent cache.
flowchart LR subgraph Resolve[Resolve] A[Manifest dependency] B[GitHubPackageDownloader] end subgraph Authorize[Authorize] C[Persistent cache checkout]:::new D[Anonymous sanitized attempt] E[AuthResolver path-scoped fallback]:::new F[GitHub Basic header environment]:::new end subgraph Cache[Cache] G[GitCache clean repository URL] H[Credential-free remote and key] end A --> B B --> C C --> D D -->|"authentication failure"| E E --> F F --> G G --> H classDef new stroke-dasharray: 5 5; class C,E,F new;Trade-offs
authorization headers; rejected URL embedding because argv, logs, remotes,
and cache identity must remain credential-free.
x-access-token:<token>encoded for HTTP Basic because Git smart HTTP andthe fixture consume that established GitHub shape.
rejected retrying every
CalledProcessErrorbecause auth failures mustreturn immediately to
AuthResolver.local Git HTTP boundary for root and sparse dependencies; retained unit tests
for fast, precise failure diagnosis.
Benefits
stored remote URLs, or Git argv values.
before the credentialed attempt, not an extra anonymous full clone.
the shared helper.
Validation
APM_E2E_TESTS=1 uv run --extra dev pytestfocused auth/cache suite:Exact-head owner evidence suite:
Canonical lint mirror:
Mutation-break evidence
1 failed in 0.51s.1 failed in 0.45s.1 failed in 0.39s.1 failed in 51.58s.All guards were restored before the exact-head passing runs.
Scenario Evidence
apm installcan populate the persistent cache for private root and subdirectory packages after one repository-scoped credential resolution.tests/integration/test_public_github_anonymous_lifecycle_e2e.py::test_private_github_fallback_normalizes_locale_and_completes_lifecycle(regression-trap for #2714)tests/integration/test_public_github_anonymous_lifecycle_e2e.py::test_private_github_fallback_normalizes_locale_and_completes_lifecycletests/unit/core/test_public_github_anonymous_first.py::test_private_github_subdirectory_cache_retries_with_scoped_credentialtests/unit/cache/test_git_cache_sparse.py::TestPartialBareFlavor::test_partial_clone_auth_failure_does_not_retry_full_clonetests/integration/test_architecture_authorities.py::test_public_github_auth_owner_guard_rejects_persistent_cache_bypassHow to test
APM_E2E_TESTS=1; expect root and subdirectory packages installed and at least two credential-free cache shards.bash scripts/lint-architecture-boundaries.sh; expect[+] architecture boundary lint clean.git config --get remote.origin.urlfor generated cache bares; expect no username or password component.Closes #2714
Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com