fix(pack): preserve remote metadata outcomes (supersedes #2661)

[Daniel Meppiel (danielmeppiel)]

fix(pack): preserve remote metadata outcomes

TL;DR

apm pack now preserves whether remote marketplace metadata was fetched, empty, unavailable, intentionally skipped, or local. Default packing remains best-effort with actionable warnings, while --strict-metadata fails before writing and --check-clean refuses to certify a degraded regeneration. This credited successor preserves the work from #2661, rebases it onto current main, and repairs the auth-fallback regression tests needed for the final CI proof.

Important

Supersedes #2661 and closes #2524. The original branch is not maintainer-writable, so this PR preserves its commits and contributor attribution instead of rewriting that branch.

Problem (WHY)

• Remote apm.yml fetch failures were collapsed into missing optional fields, so pack output could silently omit description and version.
• --check-clean could compare two equally degraded documents and report a clean marketplace even though regeneration was not certifiable.
• Strict callers had no pre-write gate or dedicated exit code for incomplete remote metadata.
• [!] The original recovery reached 99% of Shard 1 twice without producing a usable failure diagnostic, then conflicted with current main.
• [!] The recovered auth path correctly moved to AuthResolver.try_with_fallback(), but two unchanged tests still mocked only resolve() and hung in YAML parsing.

Why these matter: #2524 records the observed silent metadata loss. The recovery follows "Grounding outputs in deterministic tool execution transforms probabilistic generation into verifiable action." and keeps the change bounded because "Add what the agent lacks, omit what it knows".

Approach (WHAT)

#	Fix
1	Represent every package metadata fetch as a durable outcome with an explicit certifiability decision.
2	Prepare all selected marketplace outputs before producer writes, then enforce strict metadata as a pre-write gate.
3	Make --check-clean dry-run-only and reject uncertifiable regeneration before comparing output bytes.
4	Expose outcomes, warnings, and stable error codes in human and JSON output.
5	Keep credentials behind AuthResolver.try_with_fallback() and test both token and tokenless callback paths.

Implementation (HOW)

Surface	Files and intent
CLI contract	src/apm_cli/commands/pack.py adds --strict-metadata, exit code 5, JSON outcome reporting, and non-mutating --check-clean.
Pre-write orchestration	src/apm_cli/core/build_orchestrator.py separates marketplace preparation from production so strict validation runs before any producer writes.
Canonical outcome owner	src/apm_cli/marketplace/builder.py adds MetadataEnrichmentOutcome and MetadataEnrichmentResult, retains mapping compatibility, and records auth-backed fetch outcomes.
Drift certification	src/apm_cli/marketplace/drift_check.py adds the uncertifiable state and refuses byte comparison when remote metadata is incomplete.
CLI documentation	docs/src/content/docs/reference/cli/pack.md, docs/src/content/docs/producer/publish-to-a-marketplace.md, docs/src/content/docs/producer/releasing-from-any-ci.md, and packages/apm-guide/.apm/skills/apm-usage/commands.md document warnings, strict mode, exit codes, and CI use.
Durable release note	CHANGELOG.md records the #2524 behavior change.
Architecture boundaries	.apm/instructions/architecture.instructions.md, .github/instructions/architecture.instructions.md, and scripts/lint-architecture-boundaries.sh establish one metadata-outcome authority and prevent bypasses.
Regression coverage	tests/unit/marketplace/test_metadata_enrichment.py, tests/unit/marketplace/test_builder.py, tests/integration/test_remote_marketplace_metadata_outage_lifecycle.py, and tests/integration/test_architecture_authorities.py cover outcome states, auth callbacks, no-write behavior, lifecycle recovery, and static ownership.

Diagrams

Legend: the dashed nodes are the new decisions that prevent incomplete metadata from being mistaken for a clean marketplace.

flowchart LR
    subgraph Prepare[Prepare]
        P1[Resolve marketplace packages]
        P2[Fetch metadata outcomes]
    end
    subgraph Certify[Certify]
        C1{Metadata certifiable}
        C2[Report warnings]:::new
        C3[Fail before writes]:::new
    end
    subgraph Produce[Produce]
        W1[Write marketplace outputs]
        W2[Compare check-clean output]
    end
    P1 --> P2
    P2 --> C1
    C1 -->|yes| W1
    C1 -->|"no, default mode"| C2
    C1 -->|"no, strict mode"| C3
    C2 --> W1
    C1 -->|"no, check-clean"| C3
    W1 --> W2
    classDef new stroke-dasharray: 5 5;
    class C2,C3 new;

Loading

Trade-offs

• Warn by default; fail on request. Existing apm pack automation keeps best-effort behavior, while --strict-metadata provides an explicit fail-closed contract.
• Never certify degraded output. --check-clean rejects unavailable metadata even if regenerated bytes happen to match; a false clean result was rejected as less trustworthy.
• Prepare before writing. Metadata-dependent outputs are resolved up front in strict mode; this adds a bounded network phase but guarantees no partial producer writes.
• Keep the thread pool. Concurrent fetches remain for performance; outcomes are synchronized through one locked owner rather than serializing all metadata access.
• Use a credited successor. The original fix(pack): preserve marketplace metadata outcomes #2661 branch has maintainerCanModify=false; a successor preserves commits and attribution without mutating an inaccessible branch.

Benefits

1. Five explicit outcome states replace one ambiguous missing-value path.
2. Strict metadata failure exits 5 before any marketplace, bundle, Codex, or Claude output changes.
3. --check-clean exits 4 when metadata cannot be certified and reports the affected package.
4. JSON consumers receive metadata_enrichment.certifiable plus per-package outcomes.
5. Installed lifecycle coverage verifies repeated failure is byte-stable and restored metadata reveals real drift.

Validation

git merge-tree --write-tree 7c7e08327437b251572a8146377a0894182db667 0ede4c593b7ad24d6b45da1bade375fe7013eb6c:

62300e9faf48a5388e1eb1a33472d33f57b994ee

This confirms the credited successor applies cleanly to the exact current base.

node --test packages/apm-contributor-dashboard/tests/logic.test.mjs packages/apm-contributor-dashboard/tests/server.test.mjs:

158 tests passed

PYTHONPATH=src python3 -m pytest tests/unit/marketplace/test_metadata_enrichment.py tests/integration/test_remote_marketplace_metadata_outage_lifecycle.py -q -x --tb=short:

7 passed, 1 skipped

PYTHONPATH=src python3 -m pytest <two repaired prefetch callback tests>:

2 passed in 0.77s

Local gate and environment evidence

PASS ruff check and format (cached ruff 0.15.22)
PASS pylint R0801 (10.00/10)
PASS auth-signals, bootstrap-name authority, owner-touch detection
PASS compileall, imports, YAML I/O, file-length, portable-path, diff checks
BLOCKED exact uv Shard 1: uncached setuptools build requirement
BLOCKED canonical uv sync/lint: uncached GitPython 3.1.58

The final reserved remote CI run is required to prove the exact Shard 1 and canonical lint environments.

Scenario Evidence

#	Scenario (user promise)	Principle(s)	Test(s) proving it	Type
1	A remote metadata outage is visible instead of silently producing a trusted incomplete marketplace	DevX, Governed by policy	tests/unit/marketplace/test_metadata_enrichment.py::test_pack_json_warns_and_strict_metadata_prevents_writes (regression-trap for #2524)	unit
2	Strict packing fails before changing any producer output	Governed by policy, DevX	tests/unit/marketplace/test_metadata_enrichment.py::test_strict_metadata_preflight_prevents_all_producer_writes	unit
3	--check-clean never certifies two equally degraded marketplace documents	Governed by policy	tests/unit/marketplace/test_metadata_enrichment.py::test_drift_refuses_uncertifiable_metadata_before_comparing_output	unit
4	Repeated outage failures preserve marketplace, Codex, Claude, and bundle bytes; restored metadata exposes drift	DevX, Multi-harness support	tests/integration/test_remote_marketplace_metadata_outage_lifecycle.py::test_remote_metadata_outage_never_certifies_degraded_marketplace	integration
5	Authenticated and tokenless metadata prefetches execute through the resolver-selected fallback callback	Secure by default, Vendor-neutral	tests/unit/marketplace/test_builder.py::TestResolveGitHubToken::test_prefetch_metadata_resolves_token_before_fetching tests/unit/marketplace/test_builder.py::TestResolveGitHubToken::test_prefetch_metadata_works_without_token	unit

How to test

• Run apm pack --dry-run with remote metadata unavailable — expect success plus a package-specific metadata warning.
• Run apm pack --strict-metadata under the same outage — expect exit 5 and no changed output bytes.
• Run apm pack --check-clean under the outage — expect exit 4 with cannot certify regenerated metadata.
• Run apm pack --json --dry-run — expect metadata_enrichment.certifiable and per-package outcome entries.
• Restore metadata access and rerun --check-clean — expect actual description and version drift to be reported.

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

[Copilot]

Pull request overview

This PR tightens apm pack’s marketplace metadata handling by preserving explicit remote-metadata enrichment outcomes (fetched/empty/failed/offline/local), surfacing certifiability in JSON, and preventing --check-clean from “passing” when regeneration is based on incomplete remote metadata. It also adds a strict pre-write gate (--strict-metadata, exit 5) that fails before any producer writes when metadata cannot be certified.

Changes:

• Introduces canonical MetadataEnrichmentOutcome / MetadataEnrichmentResult in marketplace/builder.py and threads that state through pack JSON and drift checking.
• Makes --check-clean refuse byte comparison when remote metadata is uncertifiable (uncertifiable drift status; exit 4) and adds a strict pre-write gate (--strict-metadata; exit 5).
• Adds/updates unit + integration coverage, documentation, changelog entry, and an architecture boundary guard to keep metadata-certifiability ownership centralized.

Show a summary per file

File	Description
tests/unit/marketplace/test_metadata_enrichment.py	New regression tests for outcome preservation, JSON envelope, strict-mode preflight, and drift “uncertifiable” behavior.
tests/unit/marketplace/test_builder.py	Updates auth mocking to AuthResolver.try_with_fallback() and adjusts warning expectations for offline metadata enrichment.
tests/integration/test_remote_marketplace_metadata_outage_lifecycle.py	New installed-binary lifecycle test proving outage behavior is visible and non-certifying across runs.
tests/integration/test_architecture_authorities.py	Adds authority tests ensuring metadata-certifiability remains owned by marketplace/builder.py and guarded by lint.
src/apm_cli/marketplace/drift_check.py	Adds uncertifiable drift state and carries metadata enrichment result for pack JSON/error shaping.
src/apm_cli/marketplace/builder.py	Adds durable enrichment outcome/result types, preserves warnings, and routes metadata fetch through resolver-selected fallback.
src/apm_cli/core/build_orchestrator.py	Adds strict-metadata preflight and separates marketplace preparation from write to fail before mutations.
src/apm_cli/commands/pack.py	Adds --strict-metadata (exit 5), makes --check-clean non-mutating via effective dry-run, and enriches JSON with metadata certifiability/outcomes.
scripts/lint-architecture-boundaries.sh	Adds AC34 guard preventing parallel metadata-enrichment owners/consumers outside marketplace/builder.py.
packages/apm-guide/.apm/skills/apm-usage/commands.md	Documents strict-metadata and updated gate behavior/exit codes for CI usage.
docs/src/content/docs/reference/cli/pack.md	Updates CLI reference for JSON shape, --check-clean uncertifiable behavior, and --strict-metadata.
docs/src/content/docs/producer/releasing-from-any-ci.md	Updates CI release sequence to verify (--dry-run) then generate with --strict-metadata.
docs/src/content/docs/producer/publish-to-a-marketplace.md	Adds --strict-metadata to publishing guidance.
CHANGELOG.md	Adds an Unreleased “Fixed” entry for the remote-metadata outcome + gating behavior change.
.github/instructions/architecture.instructions.md	Adds marketplace metadata certifiability as a single-owner decision (builder-owned).
.apm/instructions/architecture.instructions.md	Mirrors the architecture instruction update for repo-local instructions.

Review details

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Suppressed comments (1)

src/apm_cli/marketplace/builder.py:1151

• This debug log says we're "Skipping" metadata fetch for non-GitHub hosts, but the function returns a failed outcome. Align the log message with the actual behavior so debugging doesn't imply a harmless skip.

                logger.debug(
                    "Skipping metadata fetch for %s (non-GitHub host: %s)",
                    pkg.name,
                    effective_host,
                )

• Files reviewed: 16/16 changed files
• Comments generated: 4
• Review effort level: Lite