The newest export was on disk for six days and no file here named it - #511
Merged
Conversation
Tony: "i fear our ability to document progress is faulty." The first lookup of the run he asked for proved him right. interface2 shipped `2026-09-03_revised_2v_long_STEPS_EXCLUDED` on 2026-09-03, with the field-step artifacts removed and pre-split into SENKTIDE (29) and TTX (38) cohorts. Its own README says "for any new analysis, use this folder." Nothing in this repo mentioned it, and `current_export.toml` — the file that exists BECAUSE a session could not tell which folder was current — still named the 2026-08-18 export. It is declared now, as three roles beside `default` rather than instead of it: moving `default` re-points every existing analysis and every parity fixture at a different corpus, which is a decision and not a housekeeping edit. The role notes carry what the folder's README paid to establish — that a whole-field brightness step recruits 74% (fast) and 82% (slow) of ROIs at 8.5x and 22.7x each slice's own rate, in both streams, which is the signature a coordination detector exists to find; that the pre-artifact folder is kept as a paired control; and that `io.py` still drops `on_field_step` and `field_step_id` silently, which costs nothing in an excluded folder and would cost the flagged companion everything. Three things the run could not honour until they existed. K = 10% FLOORED AT 3 was not expressible. `k_from_fraction` clamps at 1 and says in its own docstring that the clamp belongs to the caller; nobody had written the caller. The APV+CNQX+GZ pilot recorded this as a gap and got away with it because that cohort's field sizes made 10% and floor-3 the same K on every recording. Here they are not: the floor BINDS on 26 of 84 recordings, so the corpus is governed by two K rules at once. `--k-floor` now exists on `bugarach assess`, `assess_archive.py` and `derive_spec.py`, every Assessment records whether the floor bound there, and it is refused under an absolute K, where it would be a second opinion about a number somebody already chose. DERIVE_SPEC COULD ONLY WORK AT ONE ABSOLUTE K. That is right for an assessment holding every recording at every K, and wrong for one produced at a single percentage: each recording appears once, at its own count, so selecting a column would have silently dropped most of the corpus — 10% floored at 3 lands on K=3,4,5,6 here and the median column holds 55 of 84. `--k-percent` aggregates across recordings at each one's own count, which is what "one percentage, fair across the folder" means. It stamps `k_source: percentage_unreviewed`, never `mahice`: it has the shape of a K a person would set and no person set it. THE BEFORE/AFTER FIGURE HAD NO GROUP FACETS. FOUNDATIONS §9 says effects run in opposite directions by group, so a pooled panel can hide a sign change. `--per-detector --facet group` writes one page per detector with one group per column, y shared across a stream's facets and nowhere else. Two silent defects found on the way, both in code that was already shipping. The shared PNG renderer clipped at a literal `1120.0` — the old viewport as a constant — so widening the viewport widened the window and clipped back anyway. A four-facet page came out with three facets and a sliced fourth, no error, and a file that looks finished unless somebody opens it. That is the same shape as the height bug two comments above it in the same function. `make_detector_table.py` reads `rates()` by tuple position across a module boundary. Adding the group field turned it into a TypeError, which is the good outcome — a shifted numeric field would have been wrong numbers and no complaint. Full suite green (2270 passed); sapper clear. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
syncytium2
enabled auto-merge (squash)
September 9, 2026 19:18
syncytium2
added a commit
that referenced
this pull request
Sep 9, 2026
…pend what #507-509 bought (#512) The full senktide (29) and TTX (38) cohorts, artifact-excluded, assessed at K = 10% of each recording's ROI population floored at 3, simulated from their own baseline, calibrated and trained on 72 held-out simulated recordings, detected at that calibration with those trained models, and drawn as 24 pages -- twelve detectors x two treatments, four group facets x two streams each. Everything derived from the recordings is in the darkroom, per FOUNDATIONS §5. This handoff is the pointer and carries no such number; the run record beside the artifacts carries all of them. WHAT IS DIFFERENT FROM THE PILOT is most of what mattered. Both of pipeline.md's blockers closed yesterday and this run is the first to spend them: the six ran at a calibration derived from this cohort's own simulated data rather than at shipped operating points, and all six learned models ran from checkpoints in a separate process from training. Two more things the pilot could not do: background heterogeneity was FITTED from the folder rather than inherited -- eighty-four baselines where the pilot had six -- and the input was the producer's artifact-excluded export rather than a folder nobody had scanned. WHAT IT COST TO GET THERE is in #511: the newest export had been on disk six days with no file in this repo naming it, K as a percentage had no floor, derive_spec could only work at one absolute K, and the figure had no group facets and clipped its own fourth column. THE EDGE-OF-GRID ASYMMETRY is filed rather than left in a transcript. `tube` is the best learned model on this cohort and its deployed threshold sits at the top of the searched grid. The coded branch treats that as a refusal in terms -- "an optimum at the edge is not an optimum, it is the search telling you it stopped too early" -- and the learned branch warns and ships. The todo does not assume which side is right: for a probability threshold an edge may mean the model separates cleanly, and there is one measurement that tells the two apart. Nobody has run it. THE FIREFLIES NOTE IS MARKED AS A DRAFT AND MUST NOT BE SENT. It is addressed to another team, which makes it a document deliverable, and it has not been reviewed or released. Co-authored-by: richard defazio <defazio@umich.edu> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Groundwork for running the full senktide and TTX cohorts through the loop. The run itself is still going; this is everything it needed that did not exist, landed on its own so the branch does not carry code and results together.
The finding that started it
Tony asked for the full senktide and TTX data set "with excluded artifacts", and the first lookup turned up an export the repo had never heard of. interface2 shipped
2026-09-03_revised_2v_long_STEPS_EXCLUDEDsix days ago — artifacts removed, pre-split into SENKTIDE (29 recordings) and TTX (38) — and its own README says "for any new analysis, use this folder." No file here named it.current_export.toml, which exists because a session once could not tell which folder was current, still pointed at the 2026-08-18 export.It is declared now as three roles beside
default, not instead of it. Movingdefaultwould silently re-point every existing analysis and every parity fixture at a different corpus; that is a decision for Tony, and the INDEX row says so.Three things the run could not honour
k_from_fractionclamps at 1 and leaves the clamp to the caller, and nobody had written the caller--k-flooronassess,assess_archive.pyandderive_spec.py; every result records whether the floor bound therederive_specselects one absolute-K column, which drops every recording whose K landed elsewhere--k-percentaggregates across recordings at each one's own count--per-detector --facet groupThe floor is not cosmetic here. The APV+CNQX+GZ pilot filed this same gap and got away with it because that cohort's field sizes made 10% and floor-3 the same K on every recording. On this corpus the floor binds on 26 of 84 recordings, so the population is governed by two K rules at once — which is why a bound floor is recorded per recording rather than absorbed into the number.
And selecting a column would have been quiet. 10% floored at 3 resolves to K=3 on 55 recordings, 4 on 15, 5 on 12 and 6 on 2. The median column alone holds 55 of 84; the other 29 would have left the spec with nothing said about it.
k_sourcereadspercentage_unreviewed, nevermahice. MAHICE was skipped on instruction, so this K has the shape of one a person would set and no person set it, and a consumer has to be able to tell those apart.Two silent defects in code that was already shipping
The shared PNG renderer clipped at a literal
1120.0— the old viewport as a constant — so widening the viewport widened the window and clipped back anyway. A four-facet page rendered as three facets and a sliced fourth, with no error and a file that looks finished unless somebody opens it. Same shape as the height bug documented two comments above it in the same function.make_detector_table.pyreadsrates()by tuple position across a module boundary. Adding the group field turned it into aTypeError— the good outcome; a shifted numeric field would have been wrong numbers and no complaint. Fixed, with the reason written at the call site.Checks
Full suite green — 2270 passed, 46 skipped. Sapper clear,
check_quotesclear. New tests pin the floor (that it applies after the fraction meets the recording, that a bound floor is recorded rather than absorbed, that it is refused under an absolute K) and the facets (group carried unchanged, unlabelled facet drawn last, inks stable across a run's pages).🤖 Generated with Claude Code