Skip to content

fix(ci): repair stale daily release smoke fixtures - #2987

Merged
Daniel Meppiel (danielmeppiel) merged 1 commit into
mainfrom
danielmeppiel-daily-smoke-repair
Sep 15, 2026
Merged

Daniel Meppiel (danielmeppiel) merged 1 commit into
mainfrom
danielmeppiel-daily-smoke-repair

Conversation

@danielmeppiel

Copy link
Copy Markdown
Collaborator

fix(ci): repair stale daily release smoke fixtures

TL;DR

Repair the eight reported #2965 failures with changes to four test modules and a short contributor fixture note. The failures come from missing mutation cases, an obsolete GitLab mock, and Windows fixture expectations that disagree with existing production contracts. No production code, auth policy, workflow, recorded digest, or coverage threshold changes.

Important

Addresses #2965; retain the issue for a complete daily rerun. Local evidence is macOS only. The reported Linux x86 integration job was cancelled, not passed.

Problem (WHY)

Scheduled run 34927682097 and current main both use 6f3dd393ce82e4aa3b56d0de292836590af2d59e.

  • The owner-mutation registry assertion reports two missing cases: contracts-tooling-governance-evidence and onboarding-metadata-only.
  • Four GitLab REST tests supply an unconstrained mock transport plan with zero attempts. After fix(deps): preserve GitLab sparse-fetch transport #2939, REST correctly rejects that plan before the tests reach their mocked HTTP outcomes.
  • Windows discovery expects backslash paths despite canonical forward-slash output. Its HOME-only fixture also makes Path.home() disagree with Windows expanduser, producing a false install-identity collision.
  • The triage deployment test hashes raw bytes instead of the established per-file hash contract. Replacing LF with CRLF reproduces the exact recorded/actual digest pair from Windows.

The repair follows the verified-execution loop: "do the work, run a validator (a script, a reference checklist, or a self-check), fix any issues, and repeat until validation passes." Failure logs and mutation witnesses, not relaxed assertions, determine the scope.

Approach (WHAT)

# Repair
1 Add the two missing surgical mutations; keep exact once-only registry coverage and rule-specific violations.
2 Use real transport/auth owners with isolated dummy credentials and mocked Git/HTTP I/O. Fail the Git attempt before testing authorized HTTPS REST fallback.
3 Align HOME and USERPROFILE; assert canonical path serialization and byte-identical deduplication.
4 Verify deployed hashes through compute_file_hash, retaining exact installed/source equality and unchanged recorded digests. Add CRLF and content-tamper witnesses.

Implementation (HOW)

File Intent
tests/integration/test_architecture_owner_rule_mutations.py Mutate advisory authority and read-only admission semantics; no registry or linter relaxation.
tests/integration/test_download_copilot_end_to_end.py Replace only the stale GitLab fixture. Preserve all four original behavioral assertions; additionally prove Git execution/cleanup, HTTP routing, ordered fallback refs, and token-free errors.
tests/unit/adopt/test_onboarding.py Correct the platform-specific fixture/expectation and mark the two regressions windows_compat. Exercise Windows home-expansion semantics on every OS.
tests/unit/test_triage_advisory.py Use the canonical deployed-file hash and prove line-ending equivalence without accepting changed content. Add windows_compat.
docs/src/content/docs/contributing/integration-testing.md Add practical fixture guidance only, not an incident narrative.

Diagram

The highlighted fixture now reaches the existing GitLab REST behavior through a real, authorized transport attempt.

sequenceDiagram
    participant T as TestDownloadGitlabFile
    participant D as DownloadDelegate
    participant P as TransportSelector and AuthResolver
    participant G as Mock sparse Git I/O
    participant R as Mock REST I/O
    rect rgb(255, 247, 200)
        Note over T,P: Updated fixture uses real owners and isolated dummy credentials
        T->>D: download_gitlab_file with explicit HTTPS
        D->>P: Select and authenticate the Git attempt
        P-->>D: Same-origin HTTPS attempt
        D->>G: fetch_file
        G-->>D: GitFileTransportError
    end
    D->>D: Check same-origin HTTPS REST eligibility
    D->>R: Request file with scoped headers
    R-->>T: Payload or existing authentication error
    Note over T,R: Assert payload, fallback ref, callback and failure without production changes
Loading

Trade-offs

  • Repair tests against their canonical owners rather than reverting transport hardening or normalizing raw package-tree integrity. fix(cache): pin core.autocrlf=false on GitCache checkouts #2982 addresses a separate GitCache checkout problem; no files overlap.
  • Keep the existing windows_compat gate and full integration selection. No new CI job, framework, skip, threshold change, or assertion deletion.
  • No changelog/release note: this is test infrastructure, not a changed CLI contract. Native Windows/Linux and full promotion coverage remain explicitly separate evidence.

Benefits

  1. Both previously untested owner guards must now reject meaningful semantic mutations.
  2. All four GitLab outcomes are reachable without bypassing REST authorization or contacting a remote service.
  3. Three Windows-sensitive tests now run in the existing PR-time compatibility selection.
  4. CRLF differences pass the deployed-file contract; appended content still fails it.

Validation

Before: current-main matrix reproduction: 1 failed, 67 passed in 3.67s; separate GitLab reproduction: all four reported tests failed with the same empty-plan/REST-unauthorized errors as the daily run.

After, local macOS: complete touched modules plus tests/unit/deps/test_gitlab_sparse_transport_contract.py and tests/unit/test_content_hash.py:

651 passed in 163.34s (0:02:43)

Final GitLab/global-discovery/triage rerun after strengthening URL and Windows-home assertions:

71 passed in 1.52s

CI taxonomy/quality/ratchet contract selection:

90 passed in 73.61s (0:01:13)
Regression witnesses and lint evidence

In-memory mutation experiments (no production file edits):

HOME-only fixture / inconsistent Windows USERPROFILE: rejected
Aligned HOME and USERPROFILE with unchanged manifest: passed
Raw deployed-file SHA mutation: rejected by CRLF checkout witness
Canonical deployed-file hash with content-tamper witnesses: passed
Bypassing Git-first execution: rejected by fixture transport witness
Governance mutation remains valid JavaScript: passed

Canonical lint was rerun after refreshing/merging main, before commit/push: Ruff, format, pylint R0801, auth and architecture boundaries passed. The YAML/2100-line/relative-path guards also passed using equivalent Python regex checks because macOS grep lacks -P.

All checks passed!
1866 files already formatted
[+] auth-signal lint clean
[+] assertion-quality ratchet clean: AQ001=4, AQ002=12
[+] exact test duplicate ratchet clean: 1227 files, 0 allowed duplicate group(s)
YAML / 2100-line / portable-relative-path guards: PASS

mmdc validated the diagram. Docs-sync's schema-validated L0 result is no_change: tests and contributor guidance only.

Scenario evidence

Scenario Principle Test proof
Discovery preserves my existing global dependency and source bytes. Portability / DevX tests/unit/adopt/test_onboarding.py::test_global_home_reference_deduplicates and ::test_global_uses_existing_user_manifest_and_absolute_refs
GitLab fallback respects transport policy and preserves HTTP outcomes. Secure by default tests/integration/test_download_copilot_end_to_end.py::TestDownloadGitlabFile; adjacent tests/unit/deps/test_gitlab_sparse_transport_contract.py
Checkout line endings do not masquerade as tampering, but changed content does. Portability / Secure by default tests/unit/test_triage_advisory.py::test_installed_skill_files_and_recorded_hashes_match_sources
Advisory evidence grants no implementation authority; discovery stays read-only. Governed by policy tests/integration/test_architecture_owner_rule_mutations.py

Warning

The original failures are checkout unit/integration failures, not proof of a broken published artifact. Separate release-validation jobs in that run passed, but macOS ARM did not reach its later validation and Linux x86 integration was cancelled. This PR does not claim complete hosted or release-smoke success.

How to test

  • Run uv run --frozen --extra dev pytest -q tests/integration/test_architecture_owner_rule_mutations.py tests/integration/test_download_copilot_end_to_end.py tests/unit/adopt/test_onboarding.py tests/unit/test_triage_advisory.py tests/unit/deps/test_gitlab_sparse_transport_contract.py tests/unit/test_content_hash.py; expect all 651 cases to pass.
  • Run uv run --frozen --extra dev pytest -q -m windows_compat tests/unit/adopt/test_onboarding.py tests/unit/test_triage_advisory.py; expect exactly three selected cases to pass, including native Windows CI.
  • Inspect the diff: no production code, lockfile digest, workflow, or coverage threshold changes.
  • Review hosted PR checks, then confirm a complete daily integration/promotion run before treating [ci] Daily release smoke is failing #2965 as fully verified. Do not count cancelled jobs as passed.

Review contact: Daniel Meppiel (@danielmeppiel). Bounded scope is recorded in the issue artifact; no independent approval, merge, or release is claimed.

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

Fixes: #2965

Cover the two registered owner guards, exercise GitLab REST through real transport selection, and align Windows fixtures with canonical home/path and deployed-file hash contracts. Preserve production behavior and strengthen regression witnesses for #2965.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI lite review requested due to automatic review settings September 15, 2026 13:57
@danielmeppiel

Copy link
Copy Markdown
Collaborator Author

Docs sync advisory

Verdict: no_change * Pages affected: 0 * LLM calls: 0/15

The schema-validated L0 path gate matches all five changed files to tests/** or docs/** and no user-surface path. No CLI, configuration, authentication, or package format contract changes. The PR already adds directly useful portability guidance to the contributor integration-testing page; no downstream corpus edits are needed.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

No unresolved blocking issues were reported.

Review tier: Lite
Findings: None

What changed in this PR

Repairs stale CI smoke-test fixtures for issue #2965 without changing production behavior.

Changes:

  • Adds missing architecture-owner mutation coverage.
  • Repairs GitLab, Windows, and content-hash fixtures.
  • Documents portable fixture guidance.
File Summary
tests/​unit/​test_triage_advisory.py Verifies canonical hashes and CRLF invariance.
tests/​unit/​adopt/​test_onboarding.py Aligns Windows home and path expectations.
tests/​integration/​test_download_copilot_end_to_end.py Uses real transport/auth selection with mocked I/O.
tests/​integration/​test_architecture_owner_rule_mutations.py Adds coverage for two registered guards.
docs/​src/​content/​docs/​contributing/​integration-testing.md Documents portable fixture practices.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@danielmeppiel

Copy link
Copy Markdown
Collaborator Author

Hosted validation completed on commit 5e0eda1: native gh pr checks --watch exited 0. Windows Compatibility Gate: 474 passed, 2 skipped; all three repaired Windows regressions are explicitly PASSED in the log. Linux unit shards: 11,435 passed / 2 skipped and 11,412 passed / 3 skipped / 21 xfailed; both coverage floors passed. Linux lifecycle smoke: 189 passed / 1 skipped. Coverage combine, PR binary smoke, lint, architecture ratchets, CodeQL analyses, and the aggregate gate passed as well.

This is PR-time Windows/Linux evidence, not a full daily promotion rerun. The previously cancelled daily Linux x86 integration corpus remains unproven, and the full ARM/macOS promotion paths have not been rerun on this commit. No fixes beyond the original scoped commit were needed. PR remains open; nothing was merged or closed.

@danielmeppiel
Daniel Meppiel (danielmeppiel) merged commit d478988 into main Sep 15, 2026
20 checks passed
@danielmeppiel
Daniel Meppiel (danielmeppiel) deleted the danielmeppiel-daily-smoke-repair branch September 15, 2026 14:14
Copilot AI mentioned this pull request Sep 15, 2026
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[ci] Daily release smoke is failing

2 participants