The surrogate screen's machinery: twelve candidates, six controls, three yardsticks - #530
Merged
Merged
Conversation
…ree yardsticks A detector with no labels learns coordination by telling a recording from a surrogate of itself, so a surrogate that differs from real data in anything besides cross-ROI timing teaches the model the giveaway instead. On 2026-09-10 a review killed the uniform dither for exactly that: it makes within-ROI intervals shorter than any real one. Tony ruled that every replacement be measured, none pruned, and nothing shortlisted until the numbers exist. This is the tool that measures them. The plan is docs/proposals/2026-09-10-surrogate-evaluation-overnight.md, on the surrogate-screen-plan branch. The generators. src/bugarach/surrogates.py holds all twelve candidates and six controls behind one call, generate(name, trains, window, key, **params), on integer frame indices. Seven candidates come from Elephant 1.2.1, pinned as the new `surrogates` extra. Running Elephant showed it failing silently at our timescale, and the adapter corrects each failure with a test that fails if the correction is removed. The failures: floating-point time that misplaces on-grid onsets; JointISI returning sparse trains unchanged or falling back to uniform dither without saying so; a dead time silently capped, or run as plain dither at zero; no seed; and millisecond defaults. One defect not in the plan: ISI dither smooths its histogram into an integer array, rounding sparse regions to zero. The adapter hands it a float histogram smoothed by Elephant's own rule. Every Elephant call must set every parameter explicitly, and every draw is seeded from a crc32 key. assess.py's circular shift moved into a function that assess_coactivity itself calls, with the draw order unchanged, so the parity fixtures still hold. Written here rather than taken from Elephant: - Pattern jitter (Harrison & Geman 2009), built clean-room. The spec is docs/clean_room/pattern_jitter_spec.md. The primary and the adversary were each written from the spec alone. Their differential fuzz agrees exactly, and all 16 mutants are caught. - Operational-time dither (Louis et al. 2010), checked against hand-derived vectors from the paper by an agent that never read the implementation. detectors/rate.py gains train_rate, a single-train core of event_rate that gives bit-identical output. - Trial shifting. The measurements: - src/bugarach/surrogate_stats.py computes every statistic in the plan's statistics table, and the three yardsticks side by side: the mouse-split band, the paired surrogate histogram, and the exchangeable negative. - The destruction test, on a new opt-in background floor in simulate.py. With the floor off, no random numbers are drawn, so existing seeds reproduce. - src/bugarach/surrogate_discriminator.py is the per-ROI-only classifier two-sample test. It refuses permutation counts too small to reach P < alpha. - tools/build_surrogate_screen.py runs the grid, one process per cell, under a time cap and a memory cap, and writes only to the darkroom. - The generation-window rule moved into one function in assess_folder.py. - The Fano helpers moved out of a tool into src/bugarach/count_dispersion.py. Integration fix, applied while wiring the pieces together. The adapter never marked an operational-time ROI with fewer than two onsets as not estimable. The method returns such a train unchanged, and about half the real baseline ROIs are like that, so they would have been scored as preserved. They are now marked not estimable. flat_at_onset reaches the cell record, and a test pins both. Known incomplete: - CI installs .[ui,dl], which has no Elephant, so the adapter's tests skip there as a single skipped module. The rest of the new tests run without Elephant. Adding `surrogates` to the CI install lives in .github/workflows/ci.yml, which this change does not touch. - docs/INDEX.md has no rows yet for surrogate_stats, count_dispersion or build_surrogate_screen. - The discriminator is not yet wired into the CLI. - The report builder and the time-axis port are not built. - At the smallest fast J, pattern jitter and interval jitter move nothing, by construction: the window is one frame. - Joint-ISI at small J is expected to be intractable. - Operational-time's effective half-width runs to J*N/(N-1). RMS matching should use displacement_distribution(...).rms, not J. Everything here is synthetic. No real recording was read to build it, and nothing derived from one is in the tree. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…to spare The scheduler kills a cell past --cell-mem-gb and holds new cells back while the workers together pass --total-mem-gb. Both read memory with `ps -o rss=`. Git for Windows puts a Cygwin ps on PATH that has no -o, so the call failed and the except returned 0: measured on the workstation, 0 bytes for a process holding 300 MB. And os.sysconf does not exist there, so the machine read as the 16 GB fallback instead of 127 GiB, and the all-workers default was 60% of a guess. The run was about to resume on that machine with more jobs than the Mac's two; JointISI at J = 1 frame builds multi-gigabyte tables. Both now ask the Windows API through ctypes (still no psutil), keeping the working set -- the resident measure -- so a recorded 'killed at N GB resident' means one thing on every platform. After: 57 MB before a 300 MB allocation, 357 MB after; RAM 136.6 GB. The caps had never been shown to fire anywhere: the only intractable test was the in-process draw budget. New tests pin the meter to a live process and RAM to a measurement, and run the tool as a script through the real process pool until a worker is killed at each cap -- memory and time -- and recorded with the cap it hit. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… itself The surrogate-screen report draws inline SVG with no bokeh behind it, and CLAUDE.md makes the minutes-friendly axis the rule for every time axis. src/bugarach/time_axis.py ports it: AdaptiveTicker(base=60, mantissas 1/2/5/10/15/30, min_interval 1) at bokeh's default six ticks, and the hook's labels (45s, 2m, 2m30s), JavaScript quirks included -- Math.round sends halves up, numbers print in JS's shortest form. The test runs the ORIGINALS, not a transcription: BokehJS's AdaptiveTicker from the installed bokeh, in chromium, over 216 spans; and the formatter's code read out of ui/app.py and run beside the port over 580 values. Editing the hook without the port turns it red. It caught one thing a person would get wrong: for a two-hour span, 900 s and 1800 s steps are equally far from six ticks, and bokeh takes the first -- 15-minute ticks, not 20. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…could not fail tools/build_surrogate_report.py writes the plan's reports into the darkroom run folder: one page per folder and a cross-folder summary, each opening with an executive summary of what every candidate keeps and destroys, then the terms, three synthetic figures (a surrogate and its leak, what each candidate does to one ROI, the window anatomy), control power, the three yardsticks side by side, destruction, the discriminator, coverage and cost -- and the choices the verdict rule faces, each beside its measurement. No shortlist. Inline SVG only, time axes from bugarach.time_axis, a page without SVG refused, and a render gate that runs only from stamped copies of render_check.py and edge_collisions.py in the run folder and never passes by default. The first build showed every candidate at 0% flagged, uniform dither and the known-bad controls included. Measured, not a coincidence: Holm runs per cell across 65 checks, so the band's smallest P, 1/101, becomes 0.64, and the paired histogram's, 0.02 at K = 99, becomes 1.0. Neither Holm-adjusted yardstick can flag anything at these settings. Unadjusted they discriminate -- 32% of checks outside the band's 95th percentile, 38% under a raw paired P of 0.05 -- so the report says all of this first, reports unadjusted rates beside the negatives' cost, marks a candidate's paired rate unreachable where no cell passed K = 39, and puts the splits and draws a corrected test needs (1299 and 2599) at the head of the verdict-rule choices. edge_collisions.py checks only draughtsman edges and these figures have none; the gate record says it looked at nothing rather than letting a pass imply otherwise. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The surrogate screen's generators come from Elephant, an optional extra, and CI installed [ui,dl] only -- so the 98 surrogate tests skipped on every leg while the tick stayed green. The handoff owed this change; it adds the pinned 'surrogates' extra to the one install line. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
JointISI in ISI-dither mode without the square root writes its smoothed histogram into an integer array, so sparse regions truncate to zero and fall back to uniform dither. The screen's adapter already works around it (_float_smoothed_histogram); the handoff owed the entry in the todo that tracks filing all of them upstream, which stays Tony's call. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Applies the plot rule Tony restated today (#539): the K-reached lists read '49 cells at 19 draws' rather than '49 at 19', the cells column reads '6 of 6 cells', and a jitter radius on the Cossart folder reads 'frames', not 'fr'. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The run folder is in the darkroom, so Dropbox uploads every record and progress file as it changes, and on Windows a file it holds cannot be replaced: os.replace raises PermissionError (WinError 5). Twice in the 2026-09-11 run on the workstation -- 12:45 and 15:06 -- a worker died replacing its .progress file after a draw, and the cell became an error record the resume skips. _write_json now retries that one refusal with doubling waits, about 6 s in all, then lets it stand; any other error still fails at once. Tests pin both halves: refused twice then accepted, and refused every time still raising. Applied mid-run at Tony's go: workers started after this commit import it; the ones already running keep the old code, and the two error cells are rerun at the end. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… the finding Figure 5 took the median retained share across every J and every K, and on the Cossart folder it read about 0.95 for nearly every dither. Broken out, destruction climbs steeply with the jitter radius and with K, the number of ROIs the assessor requires active together: on steps_excluded's fast stream uniform dither keeps 0.96 of the planted coordination at J = 0.1 s but 0.00 at J = 1.6 s (K = 8); on the Cossart folder it still keeps 0.76 at J = 32 frames, 3.0-3.8 s. The Cossart twin has 566 ROIs, so a 50% event recruits about 283, while the scan stops at 8 -- a reading about the measure at large ROI counts, now set out among the verdict rule's choices beside those numbers. Figure 5 is now a shaded grid per stream at both ends of the K scan, every J and the value in each square; the summary tables read smallest -> largest J at the largest K. Also: the run history came from meta.json's time, and the Mac's 13:28 wrap-up had rewritten steps_excluded's meta.json after the run moved. The history now names the machine from the data-folder path meta.json records and says 'last written', not 'resumed'. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… not the candidates A blind pass over the rebuilt reports found two more blocking defects, and both had the same shape: a measurement of the scoring apparatus printed where a reader would take it for a measurement of a surrogate. The whole-window circular shift was the control that proved the destruction measure works. It is the assessor's own null - assess_coactivity draws every one of its surrogates from circular_shift_trains - so its 0.00 is true by construction and would hold if the measure were broken. It is demoted from must-pass and named as the null; homogeneous resample, which is not the null and also reads 0.00, is the honest zero-reader. Saturation was tested only at the largest jitter radius, which is the least saturated one in the sweep: the ROIs still inside the coincidence bin fall as J grows. Recomputed at every radius, 3 of 6 fast radii cannot register removal at all - so every retained span began at a radius where retention cannot fall. Those entries are marked and the arithmetic is now shown per radius. Two headline rates were averaged over a different population than the tables beside them, in the same paragraph that said those cells were excluded; they now use the tables' population. A third of the band rate came from statistics whose mouse-split band has zero width, where outside the 95 percent band degenerates to not exactly zero: 9.9 percent without them against 31.6 percent with, and both are reported. The AUC column is 0.5 x (1 + dithered share) exactly, verified on all 18 rows, and now says so. The render gate could not fail. render_check --json returns 0 before it counts anything, and on one build exited 0 while its own JSON held 104 sub-11px labels, 5 overlaps and 2 viewBox escapes. The vendored copy is stamped do-not-edit, so the caller counts and raises instead: 0/0/0 across 7,569 measured text nodes. Stale figure PNGs are deleted on rebuild, after two reviewers read three-build-old screenshots as current. 19 tests pin this. The run record is docs/reviews/report_steps_excluded_2026-09-11.md and passes both murderboard gates; its residuals say what is still wrong, including that a third blind round has not been run. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The night's Holm-adjusted yardsticks could not have flagged anything at any setting, and that was computable from the plan alone: 100 mouse splits, K at most 99, and a family of 65 checks per grid cell put the corrected floors at 0.64 and 1.00 against alpha 0.05. Two murderboard rounds on that plan missed it, because no role computes forward from declared parameters to the resolution they buy. So surrogate_stats.correction_reach() does that arithmetic, and build_surrogate_screen refuses to start when no corrected flag is reachable - naming both remedies, a bigger sample or a smaller family, with the numbers for each. --allow-underpowered exists for functional probes, which legitimately run at 3 draws. Verified firing: exit 2 on a family of 24 at 8 splits and 3 draws. Found while computing that per radius: the saturation formula multiplied the recruitment by bin/(2J+1) without capping it at 1. A probability cannot exceed 1, so the expected count cannot exceed what the event planted - yet the shipped pages reported 25.8 co-active ROIs from a twin recruiting 15.5, and 471.7 from one recruiting 283. Capped, with a test. The verdicts are unchanged; the counts were not, and the run record now says so. Also learned, and it corrects yesterday's proposal: fractional K is not a universal fix for saturation. At 10 percent it rescues Cossart, where 57 clears the 42.9 expected at J=16 frames, and makes the fast stream worse, where 3 is under the 4.6 expected at J=0.8 s - a lower floor is easier to trip. The fast twin's 31 ROIs are the constraint there, not the scan. Boards carry the probe claim: <darkroom>/bugarach/probe-surrogate-screen/, separate from the reviewed run folder. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2fa5bde added a refusal for any run whose Holm-corrected test cannot reach alpha, and it promptly fired on the tool's own fixtures: they run at 5 draws and 12 splits because they exercise plumbing, not statistical power. Four invocations now pass --allow-underpowered and say in a comment why they are entitled to. The refusal gets its own test. Without one, the same commit that argued for gates would have added a gate nothing exercises: the run must exit 2 without the flag and 0 with it, on a fixture small enough that no family size could rescue it. 2fa5bde was pushed with that suite red, and the reason is worth recording. I chained the commit behind a pipeline, so the exit status came from tail rather than pytest and a red suite read as green - the same defect as a render gate that returns 0 before it counts anything, which this branch fixed two commits ago. The 64 tests here were checked on pytest's own exit code. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The plan calls this tier required, and in the 2026-09-11 run it was void on the entire fast stream: its own negative control - real against real - flagged, so not one accuracy it reported supported any claim about any candidate. It was also underpowered by arithmetic that needed no data. At alpha 0.05 and the plan's 55 percent forced-choice effect the test needs 654 pairs for its declared 0.8 power; the windows in play gave 543, which is power 0.73. That is why the shipped tables are peppered with 'underpowered 80%, 92%, 100%' - the per-cell flags were honest, the requirement was knowable in advance. So this runs only the two controls and refuses before spending anything when the pairs available cannot reach the declared power. A refusal generates no draw. Analysis windows are cut as a duration rather than a frame count, because the Cossart folder's frame interval varies 0.0926 to 0.1190 s across sessions and the discriminator's features are counts. It is its own tool because the discriminator is not wired into build_surrogate_screen.py - which is why the 2026-09-11 run drove it from scratch scripts outside the git tree, as run_notes.json records. It reuses the grid tool's params_for rather than restating which generators take which arguments, and the screen tests' own synthetic folder rather than a second copy that could drift. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Tony's call, 2026-09-12, after the family-65 arithmetic. FOUNDATIONS section 9 wants each group's number reported beside the pooled one; that is a reporting requirement and does not make the five scopes one family. Correcting across them turned 13 statistics into 65 checks per grid cell, and at 100 mouse splits with K at most 99 the corrected floors were 0.64 and 1.00 - so every corrected rate was zero by construction and the known-bad control could not have fired. score_cell now applies Holm per scope. The family is that scope's statistics: 12 plus ks_intervals, which is the 13 the gate reports and the sample it needs is 259 splits and 519 draws rather than 1299 and 2599. meta.json's holm_family records the change, so a reader can tell which rule produced which run. The test pins both directions: each scope's adjustment equals Holm over that scope alone, and the across-scope rule would have been strictly harsher somewhere - if anyone flattens the scopes back into one family, that assertion is what fails. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Stage 2 of the probe ran at the boundary my own gate had blessed - 13 statistics, 259 mouse splits, 519 draws - and the known-bad control was flagged 7 of 13 by the band and 11 of 13 paired RAW, and 0 of 13 under Holm, with the smallest adjusted P exactly 0.0500. Holm's smallest adjusted value is m times the floor, 13 x 2/520 = 0.05 exactly, and a check fires on P < alpha, so equality fires nothing. correction_reach asked <= alpha and I had added a tolerance to let that boundary through, so it promised the one sample that cannot flag. It is strict now, and the smallest sufficient sample is 260 splits and 520 draws. The same ceil(m/alpha - 1) sat in the report builder, which is why the pages said 1299 and 2599; they are rebuilt and say 1300 and 2600. The formula existed in two modules and was wrong in both, so there is one implementation now, surrogate_stats.smallest_n, imported by the report rather than restated, with a test that brute-forces the boundary for six family sizes. Also: the report builder now reads the Holm family from the run's own meta.json where it is recorded, so a page describes the rule that produced it rather than the rule in force when the page was built. Runs before today corrected across scopes and have no such record, and fall back to counting checks per cell. And a false promise removed from surrogate_stats' own docstring. It claimed the not-estimable drop is reported as coverage. It is not: roi_mask removes those ROIs before n_roi is counted, so they leave numerator and denominator together and coverage is the share of the SCORED population. The two run opposite ways - on the 2026-09-11 fast stream joint-ISI dither excluded 1676 of 2630 ROIs and read coverage 1.000, while do-nothing excluded none and read 0.505, because the 1302 ROIs with no interval stayed in its denominator. A verdict rule taking coverage for completeness would prefer whichever generator discarded the most data. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…rs score a third of their ROIs On the fast stream every grid cell holds the same 2630 ROIs. not_estimable_rois is 0 for fifteen generators and a median 1676 for joint-ISI dither, 1664 for ISI dither and 1302 for operational-time dither, so those three are scored on 803 to 1328 ROIs while the rest are scored on all 2630. That column is in cells.csv and on no page. The quantity a reader would reach for instead points the wrong way: per-statistic coverage is 1.000 for exactly those three generators and 0.505 for every other, because the excluded ROIs leave the denominator together with the numerator. Of joint-ISI's 1676, 1452 are ROIs with fewer than three onsets - a property of this corpus - and the rest were excluded by the method itself, chiefly jisi_moved_nothing. Two consequences for the verdict rule, and they pull in opposite directions on the open cost decision: the joint-ISI family is not merely expensive on this data but largely inapplicable to it, and any rule reading coverage as completeness prefers whichever generator discarded the most data. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
At family 13 with 260 mouse splits and 520 draws the known-bad control is flagged 7 of 13 by the band and 11 of 13 paired UNDER HOLM, smallest adjusted P 0.0498 and 0.0499, while do-nothing stays at 0 of 13 with adjusted P 1.0000. The same run at 259/519 flagged 7 and 11 raw and 0 and 0 adjusted. So the corrected design does fire, which is what turns the off-by-one into a fixed defect rather than a revised opinion. Recorded too: the fast stream's discriminator tier was voided by ONE draw of an alpha-level control. Its 248 candidate rows carry a single distinct void reason naming accuracy 0.539, stamped onto 126 of them, because run_cell derives the negative control from the real features alone and it is therefore identical for every candidate in a stream. Re-run at five seeds it flags in 1 of 5, which is the false-positive rate it is built to have, while the positive control holds at 0.757 to 0.764 every time. The slow stream has zero void reasons on the same recordings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…to buy The plan's own three numbers - 100 mouse splits, K at most 99, and a Holm family of 65 checks per cell - put the smallest reachable adjusted P at 0.64 and 1.00 against alpha 0.05, so the night could not have flagged anything under correction. Correcting within scope takes the family to 13 and the required sample to 260 splits and 520 draws, and at 260/520 the known-bad control is flagged 7 of 13 and 11 of 13 under Holm while do-nothing stays at 0 of 13. At 259/519 it flags nothing. That is the whole reevaluation in one measurement. Also recorded: joint-ISI is largely inapplicable to this corpus rather than merely expensive, since 1452 of 2630 fast ROIs carry fewer than the three onsets its histogram needs; destruction works only at the radii the arithmetic allows, with homogeneous resample as an honest zero-reader where the circular shift is the assessor's own null; and the fast discriminator tier was voided by a single alpha-level control draw that reruns as 1 in 5. The recommendation is not to buy the dropped joint-ISI cells at the old settings: that compute would reproduce the J/2 binning, the no-op square-root axis, and a method scored on a third of its ROIs. Three decisions are left to Tony, including whether the joint-ISI family stays in a screen on data that cannot support it - which revisits a standing ruling and is his call, not a measurement. DRAFT: goes through /murderboard before delivery. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Eleven roles reviewed the first draft and each of its four load-bearing conclusions failed, all for one reason: it used probe numbers where production numbers existed. The probe was sized to ask whether the instrument can answer, on two recordings at three draws; it cannot support a claim about cost, capability or destruction. The withdrawn draft stays in the tree, unedited under a notice saying what failed, because the record of what was claimed is worth more than a tidy history. The correction that mattered most: the square-root axis is NOT a no-op. Twenty-three sqrt/nosqrt cell pairs in the production run differ on five to seven of thirteen statistics, and Elephant applies the square root before smoothing and before the cumulative is normalised, so normalisation cannot undo it. That claim came from the FIRST murderboard, was carried into a recommendation to delete the axis, and is now withdrawn. Other numbers re-derived from production: joint-ISI is about 300 times uniform dither on the fast stream and 27 on slow, not six; it runs on 47 of 144 slow cells, not only at the largest radius; the three excluding generators drop 47.9 to 69.5 percent of their ROIs, not a third to a half; and the rerun costs 53,432 core-hours at 19 draws or 278,409 at the plan's 99, summed from the per-radius projections each dropped cell carries in its own why column and totalled nowhere before. Two things the new draft says that the old one could not. Correcting within scope does not rescue the night: at 100 splits and 99 draws the adjusted floor is 0.129 within scope and 0.644 across, both above alpha, so only a new run can flag. And the discriminator's powered flag is computed from the data it judges, so a control that detects nothing reads powered - which is why the fast negative control looks adequate and the Cossart one, which actually detects, does not. INDEX row 125 named the superseded plan as what wins where it differs; it now names this document, flags the withdrawn draft, and a new row carries the cost and correction-scope keywords. A blind verify round on the new draft is in flight. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… lag A blind murderboard round on the surrogate-screen reevaluation had two roles hunting the same unshelved paper. One reached Gerstein 2004 open-access on ane.pl and read it; the other could not and marked the claim unverified. Both had checked the shelf first, correctly — it was simply empty of it. Neither shelved it afterwards, and neither should have. The judgment roles hold Read/Grep/Glob and cannot write; the shell-holding roles could, but are asked to keep to a declared scratch path, and a shared cross-machine shelf is not one. So the shelf's lookup half works and its deposit half has exactly one legitimate operator: the thread that adjudicates the findings. That thread had not been doing it, which is why the same download got paid for twice inside one round. Fetched and shelved under <darkroom>/bugarach/lit/surrogates/: Gerstein 2004, Acta Neurobiol Exp 64(2):203-207 (ane.pl, CC-BY) Grun et al. 2010, BMC Neurosci 11(Suppl 1):O15 (biomedcentral) Pipa et al. 2008, J Comput Neurosci 25(1):64-88 (Springer) Gerstein forces two corrections to what this project has been citing. Its own abstract calls the method "a novel variant of the dither surrogate (Date et al. 1998)" — so the dithering root is Date 1998, already on this shelf, and a trace stopping at Gerstein stops one paper short. And it says flat dither adds short intervals and lowers the peak; it does not claim intervals shorter than any real one. That stronger claim is bugarach's own measurement and is not his. Also: the Louis 2010 chapter is not unreachable. An open copy of chapter 17 is served at portal.g-node.org, which a reviewer fetched and read, so the standing "unread, paywalled" residual is wrong as written. surrogates/ and ml/ had no topic README at all, leaving fourteen PDFs in the state the shelf's top-level README calls indistinguishable from a forgotten download. Both written, with read status marked honestly as unrecorded where it cannot be established. INDEX row 124 gains the fetch routes that actually worked (publisher first; Europe PMC fullTextXML serves metadata, fullTextPDF returns 0 bytes) and the shelve-what-you-fetch rule. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ited but missing Eleven roles reviewed the v2 reevaluation blind and returned two blocking findings. Both are verified here against the production run rather than taken on a reviewer's word. "Only a new run can flag" is false. The family size is a free parameter and the draft varied it only between 65 and 13. The band floor 1/101 is actually attained -- 1,223 statistics sit exactly on it in scope `all` alone, and over a thousand in each of the four mouse groups -- so Holm on a pre-declared family of five flags on the night's existing data. This session's own gate says so in one call: correction_reach(5, 100, 99) returns band_reaches True, splits_needed 100, and the night ran exactly 100 splits. The gate built to stop this error was never asked. The draft prices a 53,432 core-hour rerun as the only option. The defect that withdrew the first draft is repeated in the second, three times, inside the section headed "with the power stated correctly". The discriminator table's slow row matches no production control. The fast negative control's verdict is inverted: production reads accuracy 0.5386, P = 0.035, significant True, which is the flag that voids 126 candidate rows and which the very next paragraph relies on. And production's 20-seed negative reruns -- fast flag_rate 0.05, slow and events 0.0 -- sat unused in the same folder while the draft argued from the probe's five seeds. Also corrected, each recomputed during adjudication: "differ on 5-7 of 13" matches no metric (delta differs on 9-13, and the Holm-corrected verdicts on zero of 13); "480 splits and 960 draws" is 520 and 1040, because paired_p is two-sided so one discordant draw takes the floor to 4/(K+1); "99 draws" was 19 for the whole ISI family; "four- to fivefold" is exactly fivefold and nothing on Cossart; "two recordings at three draws" describes stage 1 only, not the 84 recordings stages 2 and 4 ran. The record also states what this review cannot tell you: the per-role reports from the first round were not preserved, and I will not reconstruct them from memory. That is a defect in how I ran that round, not a property of the review. It carries the seeded-versus-independent marking owed to Tony. Round 1 was seeded -- I told roles the verdict, handed one a list of table candidates, gave another my own doubt, then reported what came back as discoveries. Round 2 was run blind with a minimal prompt, so its confirmation counts mean something: the missing record found by five roles independently, the inverted verdict by five, the probe-for-production table by four. Gated: roster 11 of 11 (mode standard), grants 11 ok and matching, check_quotes and sapper clear, 199 tests green. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…about to be lost Tony stopped this thread: no third draft, and the family-size decision needs discussion before it is made. This writes down everything a new session needs so none of it is repeated, and rescues the review that produced it. The eleven round-2 role reports existed only in one session's transcript. The harness wrote each agent's output to a file and every one of those files is 0 bytes. Round 1's reports were already lost exactly that way -- the run record says so and declines to reconstruct them from memory -- so round 2 is now written to docs/reviews/ as two files, roles 1-6 and 7-11, labelled with how they were recovered. Anyone running a murderboard in this tree should write the reports out as they arrive rather than trusting the task files. The handoff leads with the finding the thread ends on, because it is the one that changes what happens next: "only a new run can flag" is false. The band floor 1/101 is genuinely attained -- over a thousand statistics sit exactly on it in each of the four mouse groups -- so a pre-declared family of five flags on the night's existing data, and correction_reach(5, 100, 99) says so in one call against the 100 splits the night ran. Choosing which five now, having seen which sit on the floor, would be post-hoc. That is Tony's call and the handoff says so in those terms, lists the thirteen statistics without any performance data attached, and warns against computing which ones flagged unless he decides the post-hoc route is acceptable. Also carried forward: the seven open code findings nobody has started, the verified corrections so they are not re-derived, the literature fixed and still outstanding, and six traps this session hit -- among them a backticked span in INDEX turning the suite red, a closing backtick inside a table cell failing all eleven grant declarations, a hyphen making a Mode: line invisible, and a non-ASCII audit that passed because locale decoding left it unable to see. The root HANDOFF.md now names this as the third live thread. None of the three supersedes another; each deletes only its own file. Gated: sapper and check_quotes clear, 199 tests green, no personal paths, no stray glyphs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This was referenced Sep 12, 2026
Merged
Two conflicts, both documentation. main's versions of the lit-shelf row and the surrogate-null row were newer and supersets, and are kept. The branch's row on what the overnight screen cost pointed at the withdrawn second reevaluation and said the night could not flag at any setting, which the branch's own closing handoff contradicts; it is rewritten to start at that handoff and to mark both reevaluations withdrawn. The root HANDOFF's two banners for the detector-design and screen threads become one pointer to docs/goals/unsupervised-learning.md. Tony's stop of 2026-09-12 on the screen still stands; landing the code is not a restart. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…branch The screen's handoff, review records and withdrawn reevaluations now resolve as relative links; the decision about draft #530 leaves the waiting list because Tony made it; #531 is recorded as closed unmerged; and the CI-skips-Elephant todo closes, since the CI install line carries the surrogates extra. Landing the code is not a restart of the screen thread, and the page says so beside the code. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
syncytium2
marked this pull request as ready for review
September 14, 2026 17:13
Owner
Author
|
Landing on Tony's instruction (2026-09-14). Local suite (macOS, no Elephant): 3,019 passed. The failures were the Elephant-dependent Holm test, which CI covers now that the |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
A coordinated-event detector with no labels learns by telling a recording from a surrogate of itself. The surrogate is a resampled copy that keeps each ROI's own timing and destroys the timing between ROIs. If a surrogate differs from real data in anything else, the model learns that giveaway instead of coordination. On 2026-09-10 a review killed the uniform dither for exactly this. Tony then ruled that every replacement is measured, none is pruned, and nothing is shortlisted until the numbers exist.
This PR is the machinery that takes those measurements. The plan it implements is
docs/proposals/2026-09-10-surrogate-evaluation-overnight.mdon thesurrogate-screen-planbranch, with its review record beside it. The run and the report come after this PR, and they write to the darkroom, not here.What is here
Generators.
src/bugarach/surrogates.pyholds the twelve candidates and the six controls. They are called asgenerate(name, trains, window, key, **params)on integer frame indices, and every parameter is in frames.Seven candidates come from Elephant 1.2.1, pinned as the new
surrogatesextra. Each Elephant defect the plan lists has a correction and a test that fails without it:It also corrects one defect the plan did not list: ISI dither rounds its smoothed histogram to integers, which sends sparse ROIs back to uniform dither.
assess.py's circular shift is now a function thatassess_coactivityitself calls, with the draw order unchanged.Written here:
docs/clean_room/pattern_jitter_spec.md. The primary and the adversary were each written from the spec alone. The differential fuzz agrees exactly, and all 16 mutants are caught (tests/test_pattern_jitter.py).detectors/rate.pygainstrain_rate, which gives output bit-identical toevent_rate.Measurement:
src/bugarach/surrogate_stats.pycomputes every statistic in the plan's statistics table. It reports the three yardsticks side by side (the mouse-split band, the paired surrogate histogram, the exchangeable negative) and Holm-adjusts each grid cell's P values.simulate.py. Existing seeds reproduce.src/bugarach/surrogate_discriminator.pyis the per-ROI-only classifier two-sample test.tools/build_surrogate_screen.pyruns the grid, with time and memory caps per cell and resume on rerun. It writes only to the darkroom.src/bugarach/count_dispersion.py.Integration fix. The adapter did not mark operational-time ROIs with fewer than two onsets as not estimable. That is about half of real baseline ROIs, and each would have been scored as preserved. They are now marked, and a test pins it.
Checks
tools/sapper.py --allandtools/check_quotes.py --allfound nothing.tests/test_pattern_jitter.pypasses. The fuzz agrees and every mutant is caught.Known incomplete
.[ui,dl]. To run them there, addsurrogatesto the install line in.github/workflows/ci.yml; that file was outside this change's paths. Elephant 1.2.1 runs on Python 3.14 locally.docs/INDEX.mdneeds rows forsurrogate_stats,count_dispersionandbuild_surrogate_screen.displacement_distribution(...).rms.Every test and fixture is synthetic. No real recording was read to build this, and nothing derived from one is in the tree.
Draft, not for merge: the run phase reads real data against this tree next.
Generated with Claude Code