tests(drift): cover the missing collect_sha, and assert the sibling case positively - #89
Conversation
cargo-affected-bot
left a comment
There was a problem hiding this comment.
The test itself holds up. Traced the path it pins: with the only collect_sha pruned, check_shas_reachable puts it in missing with reachable empty, so plan yields CacheState::Miss(CacheMiss::NoReachableSha) — whose reachability() still returns Some, which is what makes status print the notice on stdout before the "no reachable collect_sha … would run all tests" line. Both asserted strings land on stdout, and the two together pin the branch: not in the repo can only come from a non-empty missing, so the generic "would run all tests" can't be satisfied by the no-DB or fingerprint-miss messages instead. The cat-file -e <sha>^{commit} guard mirrors relation_to_head exactly, so it fails for the same reason the classifier would.
One adjacent thing, outside this diff — the doc comment on the function this test now pins is stale. missing_shas_notice documents its parameter as "will rerun as 'new'" for run/collect --diff, "would rerun as 'new'" for status, but no caller passes 'new': run.rs passes "will rerun as 'stranded'", status.rs passes "would rerun as 'stranded'", and collect.rs passes a third shape the doc doesn't mention at all ("will be rerun and re-anchored at the new HEAD"). Not for this PR — it's tests-only and should stay that way — but it's a candidate for the rolling prose sweep in #88.
…widening `sibling_collect_sha_status_uses_selection` asserted only that `status` printed neither "would run all tests" nor "not in the repo". The empty-selection path prints neither either — it prints "no tests cover the changed lines and no new tests" — so the scenario stayed green through a sibling sha that anchored nothing, which is silent under-selection. Assert the two halves the name claims: the cache stayed usable (`cache=hit`) and it produced a selection (`tests would run`). Switching to `combined_output` is what reaches the summary line, which `status` writes to stderr.
cargo-affected-bot
left a comment
There was a problem hiding this comment.
The added scenario earns its place: CacheMiss::NoReachableSha — every collect_sha gone, so status widens to the full suite — is reached by no other test, and "would run all tests" is asserted positively nowhere else in the suite. Two findings on top of that, and one thing to note.
The missing-sha notice is already pinned positively. run_unions_affected_and_stranded_when_sha_is_missing asserts combined.contains("not in the repo") under the comment "The missing sha is announced, not silently dropped", and reaches that state through the same reflog-expire → gc --prune=now → cat-file -e sequence. So "nothing pinned the wording — or the emission — of the path those negatives are defined against" isn't accurate, and the PR body repeats it. What is genuinely uncovered is narrower: that scenario leaves a second sha reachable, so it exercises the partial-divergence/stranded path, whereas this one is the only test of the branch where every sha is gone. Stated as-is, a later reader who finds the diff_collect.rs assertion concludes this scenario is redundant and deletes it — losing the one branch it actually covers.
cache=hit also matches cache=hit-exact. plan.rs classifies any non-Equal relation as HitWithDivergence, so the sibling sha here is hit-with-divergence. A regression that reclassified a sibling as Equal would keep the substring and leave the scenario green — which is inside the class of silent misclassification the new assertion was added to close.
One to note rather than change here: the fifteen-line "make this commit genuinely unreachable" block is now duplicated across two scenario files, reasoning comment included. A main.rs helper is the natural home for it, but that touches diff_collect.rs and is beyond a tests-only coverage PR.
…e class
The doc comment on `missing_collect_sha_status_widens_to_all_tests` claimed
every assertion in the suite about the missing-sha notice was a negative one.
`run_unions_affected_and_stranded_when_sha_is_missing` in `diff_collect.rs`
already asserts `contains("not in the repo")` positively, and reaches that
state through the same reflog-expire/gc/cat-file sequence. State what the
scenario actually adds instead: `diff_collect.rs` covers the
partial-divergence path where a second sha survives, while this is the only
test of the branch where every collect_sha is gone.
Also tighten the sibling half's `cache=hit` to `cache=hit-with-divergence`.
`plan.rs` yields `HitWithDivergence` for any non-`Equal` relation, so the
substring `cache=hit` matched `hit-exact` too and a sibling reclassified as
`Equal` would have stayed green.
tests/functional/drift.rsis the pair of scenarios that decide whether acollect_shawhich is no longer on HEAD's lineage still anchors the diff. Both halves were asserted only negatively, so neither pinned the behavior its name claims, and both failure modes they exist to catch are ones nothing downstream reports.The sibling half asserted nothing about selection.
sibling_collect_sha_status_uses_selectionchecked thatstatusprinted neitherwould run all testsnornot in the repo. The empty-selection path prints neither either — it printsno tests cover the changed lines and no new testsand returns — so the scenario stayed green through a sibling sha that anchored nothing at all. That is silent under-selection, whichtests/CLAUDE.mdnames as the one failure mode this tool cannot detect downstream. It now asserts the two halves positively: the cache stayed usable and stayed classified as diverged (cache=hit-with-divergence, whichplan.rsreaches for any non-Equalrelation, so a sibling misclassified asEqualfails the scenario rather than passing a loosercache=hitsubstring), and it produced a selection (tests would run). Readingcombined_outputrather thanstdoutis what reaches the summary line, whichstatuswrites to stderr.The missing half had no scenario at all.
missing_collect_sha_status_widens_to_all_testscovers the branch where the onlycollect_shais gone:CacheMiss::NoReachableSha, which widensstatusto the full suite. Nothing else reaches it. The missing-sha notice on its own is already pinned positively byrun_unions_affected_and_stranded_when_sha_is_missing, but only on the partial-divergence path, where a second sha survives and the lost one's tests come back asstranded;would run all testsis asserted positively nowhere else in the suite. Resetting alone only orphans the commit — the reflog still holds it andrelation_to_headreportsReachable, which is the sibling case — so the scenario expires the reflog, prunes, and asserts the object is gone before runningstatus, or it would silently exercise the other half of the pair. That prune sequence is the same onediff_collect.rsuses, comment included; folding both into a sharedmain.rshelper is worth doing but touches a second scenario file, so it is left out of a tests-only coverage change.Verified against the live behavior rather than by reading: in the sibling scenario
statusreportscache=hit-with-divergence selection=3/3 (100%)and3 tests would run (3 affected + 0 config + 0 new + 0 stranded, 0 skipped of 3 reachable-known). Flipping each new assertion to a string the output does not contain fails the scenario, so none is vacuous.cargo testis 45 passed / 0 failed;cargo fmt --checkandcargo clippy --all-targetsare clean.Why the negatives alone could not fail
Reproducing the empty-selection path on a two-module project — fresh
collectat HEAD, clean tree —statusprints:Neither
would run all testsnornot in the repoappears, so both pre-existing assertions pass over a selection of zero. That is what the selection assertion closes, and it is why it is needed alongside the cache one rather than instead of it:cache=hit-with-divergencecatches a widening-by-cache-miss and a sibling misclassified asEqual,tests would runcatches an empty selection, and those are different regressions.mainhas moved since this branch opened, so it carries a merge ofmainas well.