New post: when did a test last fail on purpose? - #597
Merged
Conversation
Third post today, and deliberately NOT a third pass at the AI-content theme - two in a row on that would trip the cross-post repetition gate. Different cluster, different evidence, different question. External hook: SQLite carried a data-race bug for sixteen years despite a test suite larger than the database itself, and Tailscale's write-up contains the line the whole post turns on - "so rare, the SQLite developers had to add code to deliberately trigger it in their testing environments." They had to break it on purpose before any test could see it. First-hand evidence, all verified in-repo rather than recalled: - The fault-injection run: eight realistic defects planted one at a time with predictions written down first. Three caught. - Our link checker was skipping 133,874 of 149,516 links - checking about a tenth of the site and reporting the rest clean. Repointing it found five real defects that had been invisible, including a closing section offering a downloadable ROI calculator, itemised, "no email required, instant download", for a spreadsheet that never existed. - The visual suite passing while comparing against nothing, and the detail that makes it worth publishing: three earlier attempts to break it came back green and the person doing it blamed their own injections. When a check is broken, the evidence that it is broken is indistinguishable from everything being fine. Dedup: the archive's testing posts are practitioner how-tos (TDD tutorials, suite performance, mocking). None argues that a passing suite is evidence about the tests rather than the software, and none is written for someone who cannot read the code. Four questions a non-technical founder can ask, each with a checkable answer. Gates: cadence quotas measured per H2 by script - FAILED first pass in two sections (one with no sentence over 22 words, one with no single-sentence paragraph), both corrected, all six now pass. Mannerism nouns were 10 against a cap of 5 - "gate" is this post's subject matter, so five instances became "check", leaving exactly 5 in the body. Banned words zero, em dashes zero. bin/hugo-build green. marketing_copy_test 5/13/0. bin/rake test:links clean at 31,920 unique links, and the post is CONFIRMED present in the production build at _dest/public-linkcheck rather than assumed. Dated 2026-08-22, so live rather than scheduled. 1,017 words. No independent verifier ran - agent spawning unavailable this session. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016PUkwFTsiv7EB2DYKogbpg
Contributor
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
New post: when did a test last fail on purpose?
Third post today, and deliberately NOT a third pass at the AI-content theme -
two in a row on that would trip the cross-post repetition gate. Different
cluster, different evidence, different question.
External hook: SQLite carried a data-race bug for sixteen years despite a test
suite larger than the database itself, and Tailscale's write-up contains the
line the whole post turns on - "so rare, the SQLite developers had to add code
to deliberately trigger it in their testing environments." They had to break it
on purpose before any test could see it.
First-hand evidence, all verified in-repo rather than recalled:
predictions written down first. Three caught.
tenth of the site and reporting the rest clean. Repointing it found five real
defects that had been invisible, including a closing section offering a
downloadable ROI calculator, itemised, "no email required, instant download",
for a spreadsheet that never existed.
makes it worth publishing: three earlier attempts to break it came back green
and the person doing it blamed their own injections. When a check is broken,
the evidence that it is broken is indistinguishable from everything being
fine.
Dedup: the archive's testing posts are practitioner how-tos (TDD tutorials,
suite performance, mocking). None argues that a passing suite is evidence about
the tests rather than the software, and none is written for someone who cannot
read the code. Four questions a non-technical founder can ask, each with a
checkable answer.
Gates: cadence quotas measured per H2 by script - FAILED first pass in two
sections (one with no sentence over 22 words, one with no single-sentence
paragraph), both corrected, all six now pass. Mannerism nouns were 10 against a
cap of 5 - "gate" is this post's subject matter, so five instances became
"check", leaving exactly 5 in the body. Banned words zero, em dashes zero.
bin/hugo-build green. marketing_copy_test 5/13/0. bin/rake test:links clean at
31,920 unique links, and the post is CONFIRMED present in the production build
at _dest/public-linkcheck rather than assumed.
Dated 2026-08-22, so live rather than scheduled.
1,017 words.
No independent verifier ran - agent spawning unavailable this session.
Co-Authored-By: Claude Opus 5 noreply@anthropic.com
Claude-Session: https://claude.ai/code/session_016PUkwFTsiv7EB2DYKogbpg