Spec: the site derives its numbers and its measure-set, so a bench revision propagates - #412
Open
syncytium2 wants to merge 3 commits into
Open
syncytium2 wants to merge 3 commits into
syncytium2 wants to merge 3 commits into
Conversation
…vision propagates Tony, 2026-08-30: "this project is an infant... the webapp must be able to adapt to these changes. nothing is fixed, final, YET." Answering that requirement rather than the question it came out of. The scoring decision (which measures the site reports) is deliberately NOT made here and is listed under Out of scope, because making it cheap is worth more right now than making it. What is already right: learned_detector.html resolves 159 tokens from the JSON stores at build time and _page_is_current refuses to publish a stale copy. What is not, all verified on c6df955: - bakeoff.md retypes nine rows; one claim was stale eight days - README repeats them, and an orphan branch exists to hand-sync them - the front page types its own detector count: "Six" twice, "five" once, on one page, because locust is withheld from the build and nothing reconciles the two halves The withholding is already derived (_withheld_from_the_viewer reads the viewer's WITHHELD set); only the count is typed. build_site.py is pure stdlib and has no detector list, which is the blocker and is called out. Each item names a check, because prose about keeping numbers in step is the thing being replaced. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…en of them Two corrections to this spec, both from bugarach-63, both verified here against main at 0188362 before accepting. THE COUNT: it is seven reader-facing strings, not three. 171, 181, 222, 248, 779 (twice), 783, 786. Four say six, three say five. Line 283's "about six of thirty-three" is a PARTICIPATION figure and is correct; a check keyed on bare numerals would sweep it up, and that is now called out. THE FIX: deleting the count beats deriving it. A sentence with no count cannot go stale, whereas a derived count still resolves a key and can resolve the wrong one silently. It also dissolves the blocker this file spent a paragraph on -- no new data source has to be wired into a builder that imports nothing from bugarach. Prefer deleting a fact to automating it, when the fact was not load-bearing. The superseded reasoning is kept in a <details> block rather than removed, because the argument against it is the part worth reading. ONE EXCEPTION: line 181 does not just say "five", it names the roster. Deleting the numeral there still leaves five names typed by hand, and a named list is worth more to a reader than a count. _withheld_from_the_viewer() already derives WHICH detectors are withheld and is the mechanism for that line. PROVENANCE, recorded deliberately: the elimination approach arrived relayed, as Tony saying "maybe just rewrite the text so it is independent of how many detectors are currently enabled?" -- a hedged suggestion phrased as a question, relayed as "the other instruction directly". It is recorded as a suggestion. The reasoning stands on its own and would stand if he had never said it. This session mis-relayed a hedge of his as a ruling earlier today; same failure, one hop over. Also: bugarach-63 released every path and DECLINED this item on scope grounds. The paths are free; the work is not authorized. That is now stated at the bottom. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
syncytium2
pushed a commit
that referenced
this pull request
Aug 30, 2026
…see the citation Found by short-course-56 in review, not by the check. Verified before acting: the Known-traps row cited todo/2026-08-30-the-site-types-what-a-token-could- substitute.md, which exists only on site-derives-from-data (PR #412). Dead for every reader on main. The file actually there is the 2026-08-28 bake-off-page one. THE REAL DEFECT is the test. It matched only markdown ](path) links -- 18 of the 49 pointers in the file. The other 31 are bare code spans, and that set included every pointer in Known traps and docs/learned/assessment_cossart.json, the Cossart row the index was written for. The suite went green over a dead row, in the test whose own docstring argues a dead row is worse than a missing one. Third check in this repo that could not fail. Code spans are checked now. They resolve against the repo root or docs/, because the index writes both and inside a section the shorthand reads better; the job is catching a pointer to NOTHING, not enforcing one path style on prose. Five rows that used section-local shorthand are now explicit, which is better for a reader than a looser test. docs/decisions.md stays exempt by name -- owed, not written, and its row says so instead of linking. New rule in the file, because the dead row was not a typo: a row may only point at something that exists on main. This repo runs 14 worktrees; citing a branch is how the rot recurs. Land it, then index it. Two Cossart rows added, both previously unreachable: the executable spec (test_import_dandi.py, test_fair_bakeoff_transfer.py) and the 2026-08-29 transfer handoff carrying the retractions -- without which a session finds the index and still quotes superseded numbers. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
syncytium2
added a commit
that referenced
this pull request
Aug 31, 2026
* Two of the three "his" detectors were built with AI sessions, and the independence claim goes Tony, 2026-08-30: "we built the first version of loco and coact detect in matlab together." That corrects the 2026-08-29 block, which says rate_detect, coact_detect and loco_detect "are his". interface2 agrees and the dates are checkable. SpikyDetect 1-3, the synchrony lineage, are 2026-04-23/27/28 with no co-author trailer. SpikyDetect4 -> RateDetect is 2026-05-13, CoactDetect and detect_loco are both 2026-07-14, and all three carry one. The first Claude-assisted commit in interface2 is 2026-05-06: the synchrony work predates it, everything rate- and coordination-shaped comes after. Commit dates are a lower bound and Tony notes years of poor hygiene, so earlier uncommitted work is not ruled out -- what the record does settle is which commits are co-authored. THE BIGGER CHANGE is withdrawing "I was totally unaware when I designed them", on which the 08-29 block rests its "convergence, not a derivation" reading. That is real evidence from a human and is NOT available from a model co-builder, which has no introspective access to whether its output derives from CFAR-shaped training data. Tony on the lit search: "i think it surpised both of us" -- genuine, and consistent with independent convergence AND with unrecognised reconstruction. Surprise cannot separate them. Withdrawn as unsupportable, not as false. Replaced by a timeline that needs nobody's memory: built 2026-04 to 2026-07, CFAR found 2026-08-22 (ff1db4d), recognised then. That rules out deliberate derivation, which is the only charge worth pre-empting, and stops short of claiming independence from a literature this project has not searched (§7 already says so). It costs nothing that was being defended. Tony closed priority on 08-24 -- "most researchers would be kind of thrilled with the link" -- so the independence claim was never load-bearing. §5's list, port fidelity to 1e-9, the Cossart root and the Kreuz position are all untouched. Flags that docs/todo/2026-08-30-the-repo-underclaims-its-own-authorship.md predates this and overstates in its title; it must be re-read against this before anyone acts. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * The trailer says who typed, not who designed -- and "team" means a session Rewritten before review. The first version of this block tagged rate, coact and loco "AI-assisted" and withdrew §4's independence claim on that basis. Wrong twice, and Tony said so plainly: "i feel disparaged by the tag ... I did not say build three detectors for me." A Co-Authored-By trailer records who produced a commit. It carries no design attribution, and reading it as though it does disparages the person who specified the detector, ran the validation and wrote the spec into the message. interface2 shows the specification running the other way: bc833ca8 says "my CoactDetect detect_local_coincidence.m" and defines it as "the coactivity analog of RateDetect: distinct-ROI coactivity excess over a rolling LOCAL null"; 9d4d37a2 states rate's rule as "excess = primary rate (1s) - context rate (60s), Hz" -- the phrasing Tony reproduced from memory 3.5 months later, unprompted, before seeing the commit. So §4's reading stands and this block no longer disturbs it. The error is recorded rather than deleted, because the same wrong inference is available to anyone else auditing this tree. SECOND FINDING, and it is the one worth keeping: "team" in interface2 means a SESSION. Tony, 2026-08-30: "i am the only human in these repos. my teams are sessions. in those days, i'd work two workstations in parallel and call them teams." Commits say "the other team's detector #5" (bc833ca8) and "for the CoactDetect team's integration" (e8692f77). A reader outside this estate takes that as another laboratory. It is not, no outside attribution is owed, and anyone quoting interface2 commit prose publicly has to decode it first. Kept from the first draft: the timeline. Built 2026-04 to 2026-07, CFAR found 2026-08-22 (ff1db4d). Checkable in two repos, depends on nobody's memory, and now sits ALONGSIDE the recollection rather than replacing it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Coact and LoCo are one idea twice, so the CFAR count is fewer than three Tony, 2026-08-30: "we built loco and coact detect as two versions of the same circular shift idea." So bc833ca8's "the other team's detector #5" is not a rival design -- it is the same idea on his other workstation, wired in as a deliberate A/B, with the stated plan to "collapse to ONE unified detector #5 (LoCo engine + CoactDetect name)". The shipped code already says this and this document did not: src/bugarach/detectors/_shared.py:3 -- "CoactDetect and LoCo share the circular-shift surrogate machinery" -- a module that exists for that reason, with loco.py:5 naming the same core machinery. CONSEQUENCE, flagged not settled: §2 and §4 file them as two independent lineages, loco's maxlt as GO-CFAR and its percentile-of-pool as kin to OS-CFAR, coact separately. Two variants of ONE idea landing on two CFAR variants is a weaker coincidence than two detectors doing so, and the CFAR argument should not be counted twice. The arithmetic of "three of these are re-derivations" depends on how many independent arrivals there were, and it is fewer than three. Not resolving §4 here. A document that files one idea as two lineages should say so before anyone reasons from the count. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Withdraw the §4 flag: a shared null is not a shared detector Tony, 2026-08-30: "loco and coact detect performance is distinct. they are two distinct detectors with different params." Correcting this block's previous version, which read his "two versions of the same circular shift idea" as "one detector twice" and drew a false consequence -- that §4 was double-counting its CFAR mappings and there were "fewer than three independent arrivals." WITHDRAWN. §4 maps MECHANISMS, not detectors, and its rows are distinct mechanisms. Nothing in the count was wrong. What is shared is how the null is built (_shared.py, circular-shift surrogates). Everything that makes a detector differs, and the numbers were checked before saying so: coact thresholds on a significance level (alpha=1e-4, 4 params, 60s context); loco on an order statistic (99.9th percentile, 6 params, 120s context). Measured: recall 0.767 vs 0.733, probe 1.25 vs 2.50/min, detect 0.062 vs 0.248s. Two thresholding principles on a shared null, twice the false alarms, four times the runtime. That is why §4 maps them to different CFAR variants. The block now names the trap rather than deleting the error, because _shared.py invites exactly this misreading and the next reader gets it cold. Also records that "collapse to ONE unified detector #5" in bc833ca8 was a PLAN, not a record -- both ship, and the table says why. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * The ranking reversed, and the scheduled collapse would have deleted the winner Recording a piece of project history that lived only in Tony's memory. Tony, 2026-08-30: "loco kicked ass in the beginning, we might have dropped coact as the ugly sister. but now with our more serious optimization tools, coact is consistently higher than loco." bc833ca8 scheduled the A/B for deletion -- "collapse to ONE unified detector #5 (LoCo engine + CoactDetect name); detect_loco.m + the loco slot get removed then." Run on time, while LoCo was ahead, that would have kept LoCo's engine under CoactDetect's name and deleted the detector that later won, with nothing recording the swap. The scaffold outlived its own deletion notice, which is why there is anything to compare. MEASURED, from the 2026-08-29 bake-off (a510e69) which Tony had not seen when he described it from memory: coact takes 3 of 4 folds, 0.651 vs 0.638. Direction of the recollection confirmed. But "consistently higher" does NOT survive on F1: the gap is 0.013 against fold ranges of 0.61-0.71 and 0.57-0.70, and on the background sweep at 12 seeds the ordering flips with the seed block -- coact takes all seven grid points on seeds 1-12, loco takes the busy half on 13-24. F1 cannot separate them. Where coact is consistently ahead is everywhere F1 averages away: recall 0.767 vs 0.733, probe 1.25 vs 2.50/min, detect 0.062s vs 0.248s. Half the false alarms on an empty block, four times faster, more events found. So the reversal is real and the number a reader checks to confirm it reports a tie. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * An index, because the Cossart work existed and could not be found Tony, 2026-08-30: "it seems like we need an index. a file with a few key words, then a link to where the info is in the repo." The argument is the conversation that prompted it. A session spent several turns re-deriving that the Cossart/DANDI transfer machinery exists and started designing a ranking scheme around a constraint tools/import_dandi.py had already solved. Nothing was missing. It was unfindable, across 148 todos, 116 learned artifacts, 90 tools and 53 reviews. Organised by the QUESTION a session asks, not by directory. The keyword column carries the words that do NOT appear in the path -- "other lab", "fit here score there", "coincidence that is not coordination", "which folder" -- because that is where the lookups actually fail. Two rows are worth the file on their own: - transfer works by deriving a generator spec from the other lab's statistics and scoring on simulation from it, NOT by scoring on their raster. Their data is a binary raster with no coordination ground truth. Today's session got that wrong out loud before finding the importer. - "team" in interface2 commit prose means a parallel session, not another laboratory, and a stranger reading those commits will get it wrong. Carries a Known Traps section for the things that fail quietly: the worktree PYTHONPATH hazard, reading a test subset as green, a stale checkout poisoning the briefing, hand-typed tables, and three seeds being inside this bench's noise. Guarded by tests/test_index_resolves.py -- every link must resolve, and the one deliberately unresolvable pointer (docs/decisions.md, owed and not written) says so in its row rather than linking. Mutation-tested: adding a dead row fails it, removing the row passes again. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Announce the index in both places a session reads, and test that it stays announced The index had no address, which is the exact failure it was written to fix. Tony caught it: "assuming you have made the index loud". Two announcements: CLAUDE.md, which every session loads, and the gates block of session_briefing.sh, which is inside the ~1.5KB that survives a spill by design. And a test, because the first version of tests/test_index_resolves.py carried a docstring asserting "It is referenced from CLAUDE.md" while CLAUDE.md did not reference it. A claim about a check is not a check. test_the_index_is_announced now reads both files. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * The board digest was 22KB inline, and the hook that carried it spilled today Tony: "i thought this was fixed." It was, in 2026-08-20, and only for the payload that was oversized THEN. The board has since grown to 32 ACTIVE claims and the digest to 22,296B -- on its own nearly three times the harness's measured refusal threshold. The trimmed hook spilled at ~14KB in this very session: its output arrived as "Output too large (14KB). Full output saved to ..." with a 2KB preview. The degrade ladder DID fire and reported success -- "over 8000B, board digest re-rendered terse" -- and still left 15,663B. The alarm worked; the remedy did not; nothing said so. Same shape this repo keeps finding. FIX, per Tony: "break out the density of the briefing to a file with more detail. the briefing should just be pointers." The digest is still generated IN FULL and written to BOARD_DIGEST.txt beside the board it describes, so nothing is lost. The briefing carries the count, the live-worktree count, the path, and the claim-first rule. 15,663B -> 4,751B, no degrade warning. NOT CHANGED, deliberately: session_briefing.sh still renders FOUNDATIONS §9 whole at its 9,000B budget. I tried making that a pointer too and reverted it. Two tests and the file's own header say the consequence must travel with the fact -- "assert 'min_rois' in out.stdout" -- because the 2026-08-13 incident was a session that would not have been stopped by the headline alone. That file's header also records that degrading §9 to claims on every run was already considered and rejected. It was never the thing that spilled: it delivers at 8,608B under a 9,000B budget. The density question there is Tony's, and is not answered here. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Handoff: ranking the detectors, and the five decisions that are Tony's Nothing is built. A design brief, written on a context budget rather than a ruling, with a role-1 claim check and NOT murderboarded -- said in its own banner. Why the current ranking cannot be patched: F1 does not separate the detectors it is used to separate. Three independent demonstrations, all today, and the decisive one is that the background-axis winner FLIPS between two blocks of twelve seeds. Any scheme that must produce a strict order will produce a different one next week. What the brief carries that was not written down anywhere: - four failure modes are instrumented, not two. distractor_hits ("should a burst count?", open since score.py was written) has never entered a ranking. - the probe is a real axis, r(precision, probe) = -0.32, coact 0.572/1.25 vs tube 0.543/20.5 -- same precision, 17x the firings. - "false positives on empty data" is WRONG. The probe block holds 591 spikes and chance coincidence there is 12.4/min at 3 ROIs, 0.10 at 6, 0.00 at 8+. What it measures is calling a chance coincidence coordination. - Cossart transfer already works, and not by scoring on their raster: import -> assess -> derive a generator spec from THEIR statistics -> simulate with ground truth -> fair_bakeoff --score-spec. The ground-truth measures DO survive. This session asserted the opposite before finding the importer, which is why docs/INDEX.md exists. Five decisions are listed as Tony's, with why each cannot be defaulted, and four things not to do -- headed by not re-baselining the three test_background_curve asserts, which encode a claim rather than a measurement. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * The index cited a file on an unmerged branch, and the test could not see the citation Found by short-course-56 in review, not by the check. Verified before acting: the Known-traps row cited todo/2026-08-30-the-site-types-what-a-token-could- substitute.md, which exists only on site-derives-from-data (PR #412). Dead for every reader on main. The file actually there is the 2026-08-28 bake-off-page one. THE REAL DEFECT is the test. It matched only markdown ](path) links -- 18 of the 49 pointers in the file. The other 31 are bare code spans, and that set included every pointer in Known traps and docs/learned/assessment_cossart.json, the Cossart row the index was written for. The suite went green over a dead row, in the test whose own docstring argues a dead row is worse than a missing one. Third check in this repo that could not fail. Code spans are checked now. They resolve against the repo root or docs/, because the index writes both and inside a section the shorthand reads better; the job is catching a pointer to NOTHING, not enforcing one path style on prose. Five rows that used section-local shorthand are now explicit, which is better for a reader than a looser test. docs/decisions.md stays exempt by name -- owed, not written, and its row says so instead of linking. New rule in the file, because the dead row was not a typo: a row may only point at something that exists on main. This repo runs 14 worktrees; citing a branch is how the rot recurs. Land it, then index it. Two Cossart rows added, both previously unreachable: the executable spec (test_import_dandi.py, test_fair_bakeoff_transfer.py) and the 2026-08-29 transfer handoff carrying the retractions -- without which a session finds the index and still quotes superseded numbers. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * The index announcement cost 65B more than the channel had, and the pointer it duplicated comes off Two trims so this branch fits, and the second one is the point. THE ANNOUNCEMENT GOES FROM THREE LINES TO ONE. Merged with main, the briefing measured 9,215B against a 9,150B budget on a fresh clone -- over by 65 -- and the ladder degraded FOUNDATIONS §9 to its bolded claims. What it drops in that state is the consequence half: that a detector returning little under TTX is not thereby validated, and that a nonzero coactivity excess is evidence about the preparation rather than a false-alarm floor to raise min_rois until it disappears. Losing that to keep the index's origin story is the wrong trade. The story is IN the index, which is where a reader who cares already is. The address is the payload. THE POINTER FILE RETIRES INTO THE THING IT WAS STANDING IN FOR. It was added hours ago because two decisions sat in docs/learned/ where the briefing does not look, and it duplicated what docs/INDEX.md already does properly. Tony, on seeing it: "I'm beginning to realize the pointer fix was a Bandaid on a bullet wound." He is right. With the index landing, the pointer shrinks to the single decision genuinely open -- promote the 24-seed bake-off or not -- and the K question it also carried is deleted, because K was never open. THAT IS THE INCIDENT THIS BRANCH EXISTS FOR, and it happened to me. K=12 for the Cossart corpus was decided on 2026-08-29 across all 59 of their recordings, written into a handoff on main, indexed by keyword as "K=3 vs K=12 -- read before quoting any transfer figure", and the [cossart] role in current_export.toml carries DO NOT TRANSPLANT OUR K in capitals. I ran the whole transfer at k=3 and k=8 anyway, twenty-five minutes of compute, and landed it. The index's own first line says it exists because a session re-derived the Cossart machinery already in the tree. I then did it again -- because the index was not on main, because #415 was red, from the same briefing-budget defect I hit independently and fixed in #418. The fix for the failure was one unmerged pull request away, blocked by the bug that caused the failure's twin. The structural reading is filed rather than acted on: docs/todo/2026-08-31-a-decision-in-prose-will-be-re-derived.md. Short version -- the role note carries the prohibition and not the number, and cites a figure (405 ROIs) that the same handoff retracted to 566. A prohibition without an alternative is an instruction to improvise. Put k = 12 in the role, have derive_spec read it, and the error becomes impossible instead of documented. NOT MY BRANCH. bugarach-17 built #415 and ended; short-course-56 handed it over saying it was not asking me to take it on. Tony approved me landing it once it turned out to be the root cause of a wrong result. The only files touched are the one conflicted hunk and the pointer I added myself. Fresh clone, no data, no darkroom, no board: 9,056B against 9,150, no degradation, briefing tests 34 passed 1 skipped. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: richard defazio <defazio@umich.edu> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Tony, 2026-08-30: "this project is an infant. all of this will change as it matures. the webapp must be able to adapt to these changes. nothing is fixed, final, YET."
This spec decides nothing about the scoring. It makes deciding cheap and re-deciding free — which is worth more than the first guess, given the bench moved yesterday and will move again. The measure question (recall / precision / probe / timing / F1) is listed under Out of scope on purpose.
What already meets the requirement
learned_detector.htmlresolves 159 tokens from the JSON stores at build time, and_page_is_current()rebuilds it to a scratch path and compares bytes rather than trusting it. That is the pattern; the rest is applying it three more places.What does not, all verified on
c6df955bakeoff.mdThe count disagreement is not a typo.
locustis withheld from the public build, so the caption is right about the figure and the prose is right about the repo, and nothing reconciles them.The mechanism already exists and the prose doesn't use it:
_withheld_from_the_viewer()reads the viewer'sWITHHELDset rather than restating it. Only the count is typed. The blocker is called out in the spec —build_site.pyis pure stdlib and has no detector list, so it needs a source, andbakeoff.jsonis the recommended one since it already enumerates exactly what was scored.Each item names a check
Prose about keeping numbers in step is precisely what this replaces, so nothing lands as a convention. The README check is a test, not a generator — README is committed markdown, and a red suite is the alarm the orphan branch is currently standing in for.
Coordination
bugarach-63is ACTIVE ondocs/site/**,tools/build_site.pyanddocs/learned/**. This is written as a spec so it does not collide with work in flight — it is theirs to execute or hand back. The front-page count fix overlaps their authorship correction, which is rewriting the same opening prose for its own reason: the<h1>says "Six … detectors ported from MATLAB" anddocs/detector_history.mdsays only one of the six is a port.Landing this retires the orphan branch
bakeoff-table-is-a-run-behind.🤖 Generated with Claude Code