An outside read of Stella 2022 and the surrogate run, filed as eight todos - #540
Merged
Merged
Conversation
…todos Tony asked for an independent plan of action on the label-free detector thread, from a session not working on it. This is that plan, filed where open work belongs rather than as prose. Nothing here touches the thread: no writes to its branches, its PR or its run folder, all of which were read only. CI installs no Elephant, so the surrogate screen's generator tests skip on all three Python legs. The extra that would fix it exists only on the draft branch, so this closes with that branch, not before. Filed first because it is the one item that is a defect rather than a judgment. The paper answers a different question than the screen is asking. Stella's criterion is the false-positive rate of a significance test; a training contrast is a stronger requirement, because a surrogate can be conservative for a test and still hand a discriminator a marginal feature. Their reasons for preferring trial shifting are properties of a test procedure, and the trial structure it needs does not exist in continuous imaging. Their diagnosed mechanism for uniform dither is a count difference, not an interval one: dithering deletes nothing, binarization does. This project's encoder binarizes and also clips onsets onto the boundary frames, so both channels are present and one is worse here. Whether a per-ROI count of onsets surviving encoding is measured at all could not be established from the statistics module and is the first thing to check. The run's own leak results already order the candidates, and nobody has read them against the destruction results, which are on the same disk. Do nothing survives in every stream, which is what proves the leak filter is a gate and not a score; most other survivors sit at one frame of displacement, which is 0.1 s against seconds-scale coordination; and rigid shift is the only candidate surviving where displacement is comparable to real event structure. So the verdict rule needs a displacement floor derived from destruction, and rigid shift needs testing at its weak point, which is the edge it does not wrap — a count difference again. Joint-ISI dithering was dropped as intractable at every cell, and an empty row reads like a negative result. Under a no-pruning ruling that is a decision, not a resource fact. And the thread has no exit criterion. If every candidate that does not leak also fails to destroy coordination, self-supervision is not available on this data, and that is a result about the preparation worth writing up rather than a failure. The rule for reading it that way has to be written before the numbers are. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This was referenced Sep 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Tony asked for an independent plan of action on the label-free detector thread, read from outside it. Eight todos and one INDEX row. No code, and no contact with that thread — its branches, PR #530 and its darkroom run folder were read only, and nothing was written to any of them.
The one defect, filed first
docs/todo/2026-09-12-ci-installs-no-elephant-so-the-surrogate-tests-skip.md—ci.ymlinstalls.[ui,dl],tests/test_surrogates.pyopens withimportorskip("elephant"), and thesurrogatesextra pinningelephant==1.2.1exists only on the draft branch. So the adapter written to correct Elephant's silent failures is exercised on one laptop. ⚠ Closes with that branch landing, not before.What the paper does and does not license
…-stellas-criterion-is-a-significance-test-not-a-training-signal.md— their criterion is the false-positive rate of a significance test; a training contrast is strictly stronger, because a surrogate can be conservative for a test and still hand a discriminator a marginal feature. Their five reasons for preferring trial shifting are properties of a test procedure, and the trial structure it needs does not exist in continuous imaging. Their dead-time practice — per-neuron minimum ISI, capped at 4 ms, because at low rates the minimum stops being biology — independently supports this project's ruling that τ is the producer's to declare.…-count-preservation-after-encoding-is-the-gate-stella-actually-found.md— their mechanism for uniform dither's failure is that binarization deletes events, not that intervals get too short. This encoder binarizes and clips onsets onto the boundary frames, so both channels are present and one is worse here. A per-ROI count of onsets surviving encoding could not be found insurrogate_stats.pyby name; establishing whether it exists is the first task.Reading the run that already happened
…-join-the-leak-results-to-the-destruction-results.md— both halves are in the run folder and the join has not been made. Do-nothing survives the leak detector in every stream, which is what proves that detector is a gate and not a score. Most other survivors sit at one frame of displacement, 0.1 s against seconds-scale coordination. Rigid shift is the only candidate surviving out to 1.6 s fast and 1.4 s slow.…-the-verdict-rule-needs-a-displacement-floor.md— derived from destruction, not from test power, which is the defect the second review round already found once in the control gate.…-rigid-shift-without-wrap-loses-onsets-at-the-window-edge.md— truncation at the window end is an event-count difference: Stella's failure mode arriving by another door. The generation margin may already absorb it, and that should be measured rather than assumed.…-joint-isi-dithering-is-unmeasured-not-failed.md— every cell was dropped as intractable. Under a no-pruning ruling, dropping a candidate on resource limits is a decision, and an empty row reads like a negative result.…-the-label-free-detector-thread-has-no-exit-criterion.md— written before the numbers are read, or it cannot tell "not yet" from "no".Numbers quoted in the todos are aggregates read from the run's
run_summary.jsonand the discriminator metadata on 2026-09-12, each with its provenance stated in the file. No per-recording data and no recording ids.🤖 Generated with Claude Code