Skip to content

New post: when did a test last fail on purpose? - #597

Merged
pftg merged 1 commit into
masterfrom
post-test-fail-on-purpose
Aug 22, 2026
Merged

New post: when did a test last fail on purpose?#597
pftg merged 1 commit into
masterfrom
post-test-fail-on-purpose

Conversation

@pftg

@pftg pftg commented Aug 22, 2026

Copy link
Copy Markdown
Member

New post: when did a test last fail on purpose?

Third post today, and deliberately NOT a third pass at the AI-content theme -
two in a row on that would trip the cross-post repetition gate. Different
cluster, different evidence, different question.

External hook: SQLite carried a data-race bug for sixteen years despite a test
suite larger than the database itself, and Tailscale's write-up contains the
line the whole post turns on - "so rare, the SQLite developers had to add code
to deliberately trigger it in their testing environments." They had to break it
on purpose before any test could see it.

First-hand evidence, all verified in-repo rather than recalled:

  • The fault-injection run: eight realistic defects planted one at a time with
    predictions written down first. Three caught.
  • Our link checker was skipping 133,874 of 149,516 links - checking about a
    tenth of the site and reporting the rest clean. Repointing it found five real
    defects that had been invisible, including a closing section offering a
    downloadable ROI calculator, itemised, "no email required, instant download",
    for a spreadsheet that never existed.
  • The visual suite passing while comparing against nothing, and the detail that
    makes it worth publishing: three earlier attempts to break it came back green
    and the person doing it blamed their own injections. When a check is broken,
    the evidence that it is broken is indistinguishable from everything being
    fine.

Dedup: the archive's testing posts are practitioner how-tos (TDD tutorials,
suite performance, mocking). None argues that a passing suite is evidence about
the tests rather than the software, and none is written for someone who cannot
read the code. Four questions a non-technical founder can ask, each with a
checkable answer.

Gates: cadence quotas measured per H2 by script - FAILED first pass in two
sections (one with no sentence over 22 words, one with no single-sentence
paragraph), both corrected, all six now pass. Mannerism nouns were 10 against a
cap of 5 - "gate" is this post's subject matter, so five instances became
"check", leaving exactly 5 in the body. Banned words zero, em dashes zero.
bin/hugo-build green. marketing_copy_test 5/13/0. bin/rake test:links clean at
31,920 unique links, and the post is CONFIRMED present in the production build
at _dest/public-linkcheck rather than assumed.

Dated 2026-08-22, so live rather than scheduled.

1,017 words.

No independent verifier ran - agent spawning unavailable this session.

Co-Authored-By: Claude Opus 5 noreply@anthropic.com
Claude-Session: https://claude.ai/code/session_016PUkwFTsiv7EB2DYKogbpg

Third post today, and deliberately NOT a third pass at the AI-content theme -
two in a row on that would trip the cross-post repetition gate. Different
cluster, different evidence, different question.

External hook: SQLite carried a data-race bug for sixteen years despite a test
suite larger than the database itself, and Tailscale's write-up contains the
line the whole post turns on - "so rare, the SQLite developers had to add code
to deliberately trigger it in their testing environments." They had to break it
on purpose before any test could see it.

First-hand evidence, all verified in-repo rather than recalled:

- The fault-injection run: eight realistic defects planted one at a time with
  predictions written down first. Three caught.
- Our link checker was skipping 133,874 of 149,516 links - checking about a
  tenth of the site and reporting the rest clean. Repointing it found five real
  defects that had been invisible, including a closing section offering a
  downloadable ROI calculator, itemised, "no email required, instant download",
  for a spreadsheet that never existed.
- The visual suite passing while comparing against nothing, and the detail that
  makes it worth publishing: three earlier attempts to break it came back green
  and the person doing it blamed their own injections. When a check is broken,
  the evidence that it is broken is indistinguishable from everything being
  fine.

Dedup: the archive's testing posts are practitioner how-tos (TDD tutorials,
suite performance, mocking). None argues that a passing suite is evidence about
the tests rather than the software, and none is written for someone who cannot
read the code. Four questions a non-technical founder can ask, each with a
checkable answer.

Gates: cadence quotas measured per H2 by script - FAILED first pass in two
sections (one with no sentence over 22 words, one with no single-sentence
paragraph), both corrected, all six now pass. Mannerism nouns were 10 against a
cap of 5 - "gate" is this post's subject matter, so five instances became
"check", leaving exactly 5 in the body. Banned words zero, em dashes zero.
bin/hugo-build green. marketing_copy_test 5/13/0. bin/rake test:links clean at
31,920 unique links, and the post is CONFIRMED present in the production build
at _dest/public-linkcheck rather than assumed.

Dated 2026-08-22, so live rather than scheduled.

1,017 words.

No independent verifier ran - agent spawning unavailable this session.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016PUkwFTsiv7EB2DYKogbpg
@coderabbitai

coderabbitai Bot commented Aug 22, 2026

Copy link
Copy Markdown
Contributor

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: f344adf5-19a7-4075-9110-3c9b397d73b2


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@pftg
pftg merged commit dc8a9ba into master Aug 22, 2026
4 checks passed
@pftg
pftg deleted the post-test-fail-on-purpose branch August 22, 2026 15:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant