A calibration can reach the recordings it was derived from, which is the point of the loop - #507
Merged
Merged
Conversation
…the point of the loop Tony, on being told the six ran the real folder at shipped operating points while the bake-off had calibrated them on simulated data from that same cohort: *"that is the whole point of the pipeline"*. `bugarach detect --settings <detector_settings.csv>` applies a calibration, and `tools/settings_from_bakeoff.py` writes one out of a bake-off. THE FORMAT WAS ALREADY SHARED AND NOBODY HAD NOTICED. `emit.read_detector_settings` has parsed the browser's file and this module's file since it was written -- its docstring says why, in terms: "two writers produce this file ... and a file only one of them can parse is a second dialect of one table". What was missing was any caller on the way IN. So this is four columns of plumbing, not a new format, and it works both directions from the first commit. WHAT IT WAS HIDING, measured on the pilot APV+CNQX+GZ folder. At the calibration derived from that cohort's own simulated data, rate+context makes 90% more calls than at its shipped point, locust 105% more, LoCo 10% more. That is the distance between the instrument the bench scored and the instrument that ran, and until now it could not be crossed from the command line. THREE REFUSALS, AND THEY ARE FINDINGS. Only three of the six calibrate cleanly on that cohort. CoactDetect and binned SCE are refused because their folds disagreed -- a mean over a knob grid is not a knob anyone ran. SPIKE-synch is refused because every fold landed on the END of its grid, which bench.pick_operating_point already treats as a search that stopped too early. The emitter names each and leaves them out, because a file carrying shipped values for the uncalibrated detectors would read as a calibration of all six. FOUR RULES IN THE READER, each because the alternative fails quietly. A file supplies a knob and not the microscope, so grid_dt and imaging_rate_hz are recomputed from the recording AFTER the override and skipped on the way in -- a file fitted on another rig cannot impose that rig's frame interval. The fitted_ rows are provenance and reach run.json rather than a detector, where they would be a TypeError. An unknown detector, an unknown parameter or an uncoercible value is refused before the folder is walked, not on recording 84 of 85. And the three nested ports refuse a per-stream divergence by name: they read the whole recording in one RNG sequence, so there is no seam at which one stream's value could apply. The round-trip test earned its place immediately: it failed on the first run because detect writes grid_dt and imaging_rate_hz into its own settings file, so the file this module wrote would not load back into it. Also renamed the local settings dict to used_settings. It shadowed the new parameter and worked only by statement order. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This was referenced Sep 9, 2026
syncytium2
added a commit
that referenced
this pull request
Sep 9, 2026
…pend what #507-509 bought (#512) The full senktide (29) and TTX (38) cohorts, artifact-excluded, assessed at K = 10% of each recording's ROI population floored at 3, simulated from their own baseline, calibrated and trained on 72 held-out simulated recordings, detected at that calibration with those trained models, and drawn as 24 pages -- twelve detectors x two treatments, four group facets x two streams each. Everything derived from the recordings is in the darkroom, per FOUNDATIONS §5. This handoff is the pointer and carries no such number; the run record beside the artifacts carries all of them. WHAT IS DIFFERENT FROM THE PILOT is most of what mattered. Both of pipeline.md's blockers closed yesterday and this run is the first to spend them: the six ran at a calibration derived from this cohort's own simulated data rather than at shipped operating points, and all six learned models ran from checkpoints in a separate process from training. Two more things the pilot could not do: background heterogeneity was FITTED from the folder rather than inherited -- eighty-four baselines where the pilot had six -- and the input was the producer's artifact-excluded export rather than a folder nobody had scanned. WHAT IT COST TO GET THERE is in #511: the newest export had been on disk six days with no file in this repo naming it, K as a percentage had no floor, derive_spec could only work at one absolute K, and the figure had no group facets and clipped its own fourth column. THE EDGE-OF-GRID ASYMMETRY is filed rather than left in a transcript. `tube` is the best learned model on this cohort and its deployed threshold sits at the top of the searched grid. The coded branch treats that as a refusal in terms -- "an optimum at the edge is not an optimum, it is the search telling you it stopped too early" -- and the learned branch warns and ships. The todo does not assume which side is right: for a probability threshold an edge may mean the model separates cleanly, and there is one measurement that tells the two apart. Nobody has run it. THE FIREFLIES NOTE IS MARKED AS A DRAFT AND MUST NOT BE SENT. It is addressed to another team, which makes it a document deliverable, and it has not been reviewed or released. Co-authored-by: richard defazio <defazio@umich.edu> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Tony, told that the six ran the pilot folder at shipped operating points while the bake-off
had calibrated them on simulated data derived from that same cohort: "that is the whole
point of the pipeline".
bugarach detect --settings <detector_settings.csv>applies a calibration.tools/settings_from_bakeoff.pywrites one out of a bake-off.The format was already shared and nobody had noticed
emit.read_detector_settingshas parsed the browser's file and this module's file since itwas written — its docstring says why, in terms: "two writers produce this file … and a file
only one of them can parse is a second dialect of one table." What was missing was any
caller on the way in. So this is plumbing, not a new format, and it works in both
directions from the first commit.
What it was hiding
Measured on the pilot APV+CNQX+GZ folder, at the calibration derived from that cohort's own
simulated data:
That is the distance between the instrument the bench scored and the instrument that ran.
Three refusals, and they are findings
Only three of the six calibrate cleanly on that cohort.
CoactDetectandbinned SCEarerefused because their folds disagreed — a mean over a knob grid is not a knob anyone ran.
SPIKE-synchis refused because every fold landed on the end of its grid, whichbench.pick_operating_pointalready treats as a search that stopped too early. Each is namedand left out, because a file carrying shipped values for the uncalibrated detectors would
read as a calibration of all six.
Four rules in the reader
A file supplies a knob and not the microscope —
grid_dtandimaging_rate_hzarerecomputed from the recording after the override, so a file fitted on another rig cannot
impose that rig's frame interval.
fitted_*rows are provenance and reachrun.jsonratherthan a detector, where they would be a
TypeError. A bad file is refused before the folderis walked, not on recording 84 of 85. And the three nested ports refuse a per-stream
divergence by name — they read the whole recording in one RNG sequence, so there is no
seam at which one stream's value could apply.
The round-trip test earned its place immediately: it failed on the first run because
detectwrites
grid_dtandimaging_rate_hzinto its own settings file, so the file this modulewrote would not load back into it.
Full suite green: 2220 passed, 46 skipped, 1 xfailed.
docs/pipeline.md's blocker listis down from two to one — model persistence is next.
🤖 Generated with Claude Code