Skip to content

Fault-inject both directions: the rule was half-written - #609

Merged
pftg merged 1 commit into
masterfrom
kaizen-two-sided
Aug 22, 2026
Merged

Fault-inject both directions: the rule was half-written#609
pftg merged 1 commit into
masterfrom
kaizen-two-sided

Conversation

@pftg

@pftg pftg commented Aug 22, 2026

Copy link
Copy Markdown
Member

Fault-inject both directions: the rule was half-written

Kaizen pass over today. The dominant failure was not slop or wrong numbers - it
was building an instrument, trusting its verdict, and nearly changing correct
work. Four times:

case-study marker accused our own real jt_site case study
generalized-actor rule fired on "anyone on the team could remember"
phrase marker flagged a quoted exhibit inside a ```diff as our claim
single-line img grep reported a working diagram as not rendering

One rule already covers half of this - "a new test isn't done until you've
BROKEN THE CODE and watched it fail". It is one-sided. It says inject the defect
and watch it fail; it never says inject a legitimate LOOKALIKE and watch it
pass. All four misses were that missing half.

Smallest change that prevents the class: one clause on the rule that already
exists, rather than a new rule or a new tool.

Verified rather than asserted, on the ratchet I modified an hour ago:

legitimate exhibit (quoted bad writing inside a fence) -> 0 failures PASS
the real claim (same phrase, in prose) -> Expected 10 to be <= 9 FAIL

Both directions confirmed on a throwaway probe post, which was then deleted;
suite back to 5 runs / 13 assertions / 0 failures. That is the first gate today
tested in both directions, and it is the one that had a false positive this
afternoon.

The operational half of the clause is the part that would have saved the time:
when a gate accuses something, reproduce the accusation by hand before
believing it.
Three of the four above survived because the verdict looked
authoritative and I acted on it instead of checking it - the same shape as
trusting a green run, inverted.

Kept deliberately small. No new script, no new gate, no framework - one clause
on an existing BLOCKING rule, with the four cases named so the next reader knows
it is describing something that happened rather than a hypothetical.

Gates: marketing_copy_test 5/13/0. bin/hugo-build green. Instruction-layer diff.

Co-Authored-By: Claude Opus 5 noreply@anthropic.com
Claude-Session: https://claude.ai/code/session_016PUkwFTsiv7EB2DYKogbpg

Kaizen pass over today. The dominant failure was not slop or wrong numbers - it
was building an instrument, trusting its verdict, and nearly changing correct
work. Four times:

  case-study marker      accused our own real jt_site case study
  generalized-actor rule fired on "anyone on the team could remember"
  phrase marker          flagged a quoted exhibit inside a ```diff as our claim
  single-line img grep   reported a working diagram as not rendering

One rule already covers half of this - "a new test isn't done until you've
BROKEN THE CODE and watched it fail". It is one-sided. It says inject the defect
and watch it fail; it never says inject a legitimate LOOKALIKE and watch it
pass. All four misses were that missing half.

Smallest change that prevents the class: one clause on the rule that already
exists, rather than a new rule or a new tool.

Verified rather than asserted, on the ratchet I modified an hour ago:

  legitimate exhibit (quoted bad writing inside a fence)  -> 0 failures  PASS
  the real claim (same phrase, in prose)                  -> Expected 10 to be <= 9  FAIL

Both directions confirmed on a throwaway probe post, which was then deleted;
suite back to 5 runs / 13 assertions / 0 failures. That is the first gate today
tested in both directions, and it is the one that had a false positive this
afternoon.

The operational half of the clause is the part that would have saved the time:
**when a gate accuses something, reproduce the accusation by hand before
believing it.** Three of the four above survived because the verdict looked
authoritative and I acted on it instead of checking it - the same shape as
trusting a green run, inverted.

Kept deliberately small. No new script, no new gate, no framework - one clause
on an existing BLOCKING rule, with the four cases named so the next reader knows
it is describing something that happened rather than a hypothetical.

Gates: marketing_copy_test 5/13/0. bin/hugo-build green. Instruction-layer diff.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016PUkwFTsiv7EB2DYKogbpg
@coderabbitai

coderabbitai Bot commented Aug 22, 2026

Copy link
Copy Markdown
Contributor

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: f5d9e6ca-c46b-44b6-b0d5-0f8d14927246


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@pftg
pftg merged commit 1ca5e78 into master Aug 22, 2026
@pftg
pftg deleted the kaizen-two-sided branch August 22, 2026 18:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant