claude-code-hooks: pitfall #28 — the fallback target that's right for one reason and wrong for another - #253
Closed
daymade wants to merge 3 commits into
Closed
claude-code-hooks: pitfall #28 — the fallback target that's right for one reason and wrong for another#253daymade wants to merge 3 commits into
daymade wants to merge 3 commits into
Conversation
…r that failed calibration Adds a numbers section to SKILL.md plus scripts/scan_numeric_consistency.py. The category matters because the dictionary structurally cannot reach it: a dictionary rule needs a stable wrong->right mapping, and damaged digits have none — one replacement corrupts "21", "5+1" and a date into three different shapes. Three sub-classes get distinct settlement paths (magnitude, dropped measure word, polarity inversion); polarity is called out as the dangerous one and deliberately left un-automated, because the sentence stays grammatical and only the meaning reverses. The scanner's needle list is the dictionary's own to_text values — the strings this toolchain writes into transcripts are exactly the ones that must never sit inside a number, so no hand-maintained list is needed. Three constraints came from calibration against a 222-transcript corpus rather than from design, and each is documented where enforced: bounded 2-char widening (unbounded fired 97x on healthy text vs 46 real), digit-before-term only (the mirror direction is just how Chinese is written), metadata lines skipped. Final: all four damage shapes caught, 0.004 findings/file. CHANGELOG also records a detector that was built, calibrated and DELETED: the magnitude-disagreement check fired 280 times with one true positive, because a speaker contrasting two magnitudes is arithmetically identical to a dropped zero. Narrowing on restatement markers made it worse. The negative result is the reusable part — that discrimination needs the semantic read the native AI pass already does, not lexical rules. Independent review (fresh-context agent, artifact + reader spec only) raised 12 findings; 10 fixed, 1 kept with an explicit caveat, 1 declined. Notably it caught that (?<![\w.]) matches CJK in Python 3, so "总共30+家" scanned clean while "total 30+ customers" matched — invisible on this corpus because ASR put spaces around numbers. Also fixed: silent clean report on a missing/corrupt dictionary (now exit 2), exit-code collision between "crashed" and "found candidates", comma-separated --domain matching the convention taught elsewhere, and unsourced error-rate percentages replaced with a verifiable qualitative claim. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… reproducible test suite
Second independent-review pass confirmed the earlier fixes and left four items.
Three are addressed here; the fourth (ROVER attribution unverified by the
reviewer) needs no code change — the citation carries name, institution and
year, which is what makes it checkable.
- The calibration claim was the one real gap: "222 transcripts, 0.004/file"
cannot be re-run by a reader, because the corpus is private and cannot ship.
Replaced by scripts/tests/test_numeric_consistency.py (22 cases), which pins
the *behaviour* that measurement bought instead of the number: every damage
shape is detected, and the healthy-input shapes that killed two earlier
versions of this scanner stay silent. SKILL.md now points at the suite and
says plainly which part is not reproducible and why.
- --json plus a hard failure printed nothing on stdout, so a caller parsing it
unconditionally got an empty string and had to guess whether that meant
"clean". It now emits {"_scan": {"error": ..., "needles_loaded": 0}} and
still exits 2.
- "fetch_minute_audio.py gets you the audio to check that timestamp" left the
last step unstated. Rewritten to route through the path this skill already
has — wire the audio: frontmatter, enqueue the number, press Q in the review
dashboard to hear that exact utterance — rather than implying a second
mechanism.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… for one reason and wrong for another A verification hook derives its target from the command text and falls back to the event cwd when it can't parse one. That fallback is correct for one of the two reasons it fires and wrong for the other, and the code path is identical: `git push` (no explicit target) legitimately adopts the cwd; `git -C "$R" push` *names* a target the hook cannot resolve, adopts the same cwd, and renders a confident verdict about a different repository. Both render byte-identically (MD5-equal, measured), so neither hook nor reader can separate the honest verdict from the misbound one. Framed as the checking-tool form of the confused deputy problem (Hardy 1988): a component with authority to answer becomes confused about which object it is answering for. Fix: a fallback value must carry the reason it was chosen, and only "no explicit target" earns a verdict. Includes the assumption that hid it — both hooks carried a comment asserting this path could only fail loudly "because the wrong path just fails to read". That holds only if the fallback lands somewhere invalid; in a multi-repo session it lands in another *valid* repo, the read succeeds, and returns a real and entirely irrelevant fact. The false green grew out of the belief that a false green was impossible. Also, from the independent review: - Meta-principle triage list indexed #1–#22 only; #23–#28 were unreachable by the document's own stated method ("match the symptom first"). Added routing for all six. - #10 now forward-references #28 — #10's own generalization lists uninterpolated $VARS as in-class, but its fix (expanduser) is a no-op for them, so a reader stopping at #10 believes the class is covered. - Noted the shared-library reassurance's boundary: it protects hooks that *source* the helper; one carrying its own inline parser inherits none of it, which is how #28 survived #10. - Alert-fatigue figures given a searchable anchor instead of bare "published work". - #28 now names the worked-example hooks; an entry you can't go read a before/after for is not verifiable. - SKILL.md: removed a dangling "that table" whose nearest antecedent was an unrelated table in that file. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Owner
Author
|
分支基线过时导致冲突（含一笔已随 #252 合并的重复 commit）。同内容用干净分支重开，见下方新 PR。 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
起因
本 session 里
git-push-verify/git-commit-headcheck连续五次报了错仓的 HEAD，措辞却是「权威源，勿信命令行内输出」——即它一边给出关于另一个仓库的正确事实，一边要求我放弃自己的观察。每次我都用git -C显式复核才没被带偏。机制（喂真实 JSON 端到端测出来的，不是读代码推断）
git -C /字面路径 pushR=/路径; git -C $R pushgit push（无-C）第 2 行与第 3 行输出逐字节相同（MD5 相等，独立评审与我各自测过）。根因：「命令没有显式目标」和「有显式目标但值读不到」退回了同一个兜底值，于是同一句 ✅ 既可能是最强判定、也可能是关于另一个仓的正确废话。而路径存在 shell 变量里根本不是奇技——恰恰是多仓 session 的常态，也就是这个缺陷最咬人的场景。
抽象出来的原则：兜底值必须携带「我为什么用兜底」的原因，因为不同原因下同一个值的可信度不同。
最该记的一层：藏住它的那个假设
两个 hook 的源码里都写着注释，断言这条路径「失效方向是 fail-loud，没有假绿路径」——因为「回退后的路径会读不到 HEAD」。这个判断被实测推翻：多仓 session 里回退到的事件 cwd 通常是另一个有效 git 仓，
git log成功，返回一个真实、可核验、完全无关的事实。假绿恰恰从「不可能假绿」这个假设里长出来。领域锚（两条都不在原文件里）
不只是文档 —— 参考实现也修了
独立评审的头条发现是：我写的处方，被描述的那个 hook 并没有实现，而且它当时的行为正是处方第 2 条点名的反模式（把免责句放在 ✅ 之后）。文档在说一件实现没做的事，读者去看实现会抄到不合规版本。
所以两个生产 hook 一并修了（私有仓
daymade/scripts，commit41c4227）：给兜底标原因，「有显式目标但读不到」拒绝作答，且不借用「权威源 / 以本行为准」的措辞——那是测量挣来的，被绑定花掉；绑定是猜的时候不该借用。四形态端到端回归零误伤，并已在本 session 的真实调用路径上自证（变量形态→未核对，字面路径→正确判定）。其余 findings（全部来自独立评审，已复现后修）
#1–#22，#23–#28六条按文件自称的方法（「先匹配 symptom」）根本到不了。补齐全部六条的路由。$VARS列为同类，但它给的修法（expanduser）对$VAR是 no-op——读者停在 Add perf-audit to use cases section #10 会以为这一类已覆盖。that table的最近先行词是本文件里另一张无关的表。闸门
bash -n×2 + 四形态真实 JSON 端到端 + hook-health-check 静默通过评审跑的是修复前的版本；上述文档修改与两个 hook 的改动都没有再过一轮 fresh agent。我做了实跑验证，但那是自审。所以没有自己合并——请你决定是先补一轮还是直接合。
🤖 Generated with Claude Code