Skip to content

tests(drift): cover the missing collect_sha, and assert the sibling case positively - #89

Merged
max-sixty merged 4 commits into
mainfrom
tests/missing-collect-sha-widening
Sep 21, 2026
Merged

max-sixty merged 4 commits into
mainfrom
tests/missing-collect-sha-widening

Conversation

@cargo-affected-bot

@cargo-affected-bot cargo-affected-bot commented Aug 22, 2026

Copy link
Copy Markdown
Collaborator

tests/functional/drift.rs is the pair of scenarios that decide whether a collect_sha which is no longer on HEAD's lineage still anchors the diff. Both halves were asserted only negatively, so neither pinned the behavior its name claims, and both failure modes they exist to catch are ones nothing downstream reports.

The sibling half asserted nothing about selection. sibling_collect_sha_status_uses_selection checked that status printed neither would run all tests nor not in the repo. The empty-selection path prints neither either — it prints no tests cover the changed lines and no new tests and returns — so the scenario stayed green through a sibling sha that anchored nothing at all. That is silent under-selection, which tests/CLAUDE.md names as the one failure mode this tool cannot detect downstream. It now asserts the two halves positively: the cache stayed usable and stayed classified as diverged (cache=hit-with-divergence, which plan.rs reaches for any non-Equal relation, so a sibling misclassified as Equal fails the scenario rather than passing a looser cache=hit substring), and it produced a selection (tests would run). Reading combined_output rather than stdout is what reaches the summary line, which status writes to stderr.

The missing half had no scenario at all. missing_collect_sha_status_widens_to_all_tests covers the branch where the only collect_sha is gone: CacheMiss::NoReachableSha, which widens status to the full suite. Nothing else reaches it. The missing-sha notice on its own is already pinned positively by run_unions_affected_and_stranded_when_sha_is_missing, but only on the partial-divergence path, where a second sha survives and the lost one's tests come back as stranded; would run all tests is asserted positively nowhere else in the suite. Resetting alone only orphans the commit — the reflog still holds it and relation_to_head reports Reachable, which is the sibling case — so the scenario expires the reflog, prunes, and asserts the object is gone before running status, or it would silently exercise the other half of the pair. That prune sequence is the same one diff_collect.rs uses, comment included; folding both into a shared main.rs helper is worth doing but touches a second scenario file, so it is left out of a tests-only coverage change.

Verified against the live behavior rather than by reading: in the sibling scenario status reports cache=hit-with-divergence selection=3/3 (100%) and 3 tests would run (3 affected + 0 config + 0 new + 0 stranded, 0 skipped of 3 reachable-known). Flipping each new assertion to a string the output does not contain fails the scenario, so none is vacuous. cargo test is 45 passed / 0 failed; cargo fmt --check and cargo clippy --all-targets are clean.

Why the negatives alone could not fail

Reproducing the empty-selection path on a two-module project — fresh collect at HEAD, clean tree — status prints:

cargo-affected: cache=hit-exact selection=0/3 (0%)
coverage database: target/affected/coverage.db
...
no changes since the newest collect_sha and no new tests — nothing would run

Neither would run all tests nor not in the repo appears, so both pre-existing assertions pass over a selection of zero. That is what the selection assertion closes, and it is why it is needed alongside the cache one rather than instead of it: cache=hit-with-divergence catches a widening-by-cache-miss and a sibling misclassified as Equal, tests would run catches an empty selection, and those are different regressions.

main has moved since this branch opened, so it carries a merge of main as well.

@cargo-affected-bot cargo-affected-bot left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test itself holds up. Traced the path it pins: with the only collect_sha pruned, check_shas_reachable puts it in missing with reachable empty, so plan yields CacheState::Miss(CacheMiss::NoReachableSha) — whose reachability() still returns Some, which is what makes status print the notice on stdout before the "no reachable collect_sha … would run all tests" line. Both asserted strings land on stdout, and the two together pin the branch: not in the repo can only come from a non-empty missing, so the generic "would run all tests" can't be satisfied by the no-DB or fingerprint-miss messages instead. The cat-file -e <sha>^{commit} guard mirrors relation_to_head exactly, so it fails for the same reason the classifier would.

One adjacent thing, outside this diff — the doc comment on the function this test now pins is stale. missing_shas_notice documents its parameter as "will rerun as 'new'" for run/collect --diff, "would rerun as 'new'" for status, but no caller passes 'new': run.rs passes "will rerun as 'stranded'", status.rs passes "would rerun as 'stranded'", and collect.rs passes a third shape the doc doesn't mention at all ("will be rerun and re-anchored at the new HEAD"). Not for this PR — it's tests-only and should stay that way — but it's a candidate for the rolling prose sweep in #88.

cargo-affected-bot and others added 2 commits September 21, 2026 07:06
…widening

`sibling_collect_sha_status_uses_selection` asserted only that `status`
printed neither "would run all tests" nor "not in the repo". The
empty-selection path prints neither either — it prints "no tests cover the
changed lines and no new tests" — so the scenario stayed green through a
sibling sha that anchored nothing, which is silent under-selection.

Assert the two halves the name claims: the cache stayed usable
(`cache=hit`) and it produced a selection (`tests would run`). Switching to
`combined_output` is what reaches the summary line, which `status` writes to
stderr.
@cargo-affected-bot cargo-affected-bot changed the title tests: positively assert the missing-collect_sha notice and widening tests(drift): cover the missing collect_sha, and assert the sibling case positively Sep 21, 2026

@cargo-affected-bot cargo-affected-bot left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The added scenario earns its place: CacheMiss::NoReachableSha — every collect_sha gone, so status widens to the full suite — is reached by no other test, and "would run all tests" is asserted positively nowhere else in the suite. Two findings on top of that, and one thing to note.

The missing-sha notice is already pinned positively. run_unions_affected_and_stranded_when_sha_is_missing asserts combined.contains("not in the repo") under the comment "The missing sha is announced, not silently dropped", and reaches that state through the same reflog-expire → gc --prune=nowcat-file -e sequence. So "nothing pinned the wording — or the emission — of the path those negatives are defined against" isn't accurate, and the PR body repeats it. What is genuinely uncovered is narrower: that scenario leaves a second sha reachable, so it exercises the partial-divergence/stranded path, whereas this one is the only test of the branch where every sha is gone. Stated as-is, a later reader who finds the diff_collect.rs assertion concludes this scenario is redundant and deletes it — losing the one branch it actually covers.

cache=hit also matches cache=hit-exact. plan.rs classifies any non-Equal relation as HitWithDivergence, so the sibling sha here is hit-with-divergence. A regression that reclassified a sibling as Equal would keep the substring and leave the scenario green — which is inside the class of silent misclassification the new assertion was added to close.

One to note rather than change here: the fifteen-line "make this commit genuinely unreachable" block is now duplicated across two scenario files, reasoning comment included. A main.rs helper is the natural home for it, but that touches diff_collect.rs and is beyond a tests-only coverage PR.

Comment thread tests/functional/drift.rs
Comment thread tests/functional/drift.rs Outdated
…e class

The doc comment on `missing_collect_sha_status_widens_to_all_tests` claimed
every assertion in the suite about the missing-sha notice was a negative one.
`run_unions_affected_and_stranded_when_sha_is_missing` in `diff_collect.rs`
already asserts `contains("not in the repo")` positively, and reaches that
state through the same reflog-expire/gc/cat-file sequence. State what the
scenario actually adds instead: `diff_collect.rs` covers the
partial-divergence path where a second sha survives, while this is the only
test of the branch where every collect_sha is gone.

Also tighten the sibling half's `cache=hit` to `cache=hit-with-divergence`.
`plan.rs` yields `HitWithDivergence` for any non-`Equal` relation, so the
substring `cache=hit` matched `hit-exact` too and a sibling reclassified as
`Equal` would have stayed green.
@max-sixty
max-sixty merged commit d30101d into main Sep 21, 2026
6 checks passed
@max-sixty
max-sixty deleted the tests/missing-collect-sha-widening branch September 21, 2026 08:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants