Skip to content

An outside read of Stella 2022 and the surrogate run, filed as eight todos - #540

Merged
syncytium2 merged 1 commit into
mainfrom
file-the-surrogate-screen-todos
Sep 12, 2026
Merged

syncytium2 merged 1 commit into
mainfrom
file-the-surrogate-screen-todos

Conversation

@syncytium2

Copy link
Copy Markdown
Owner

Tony asked for an independent plan of action on the label-free detector thread, read from outside it. Eight todos and one INDEX row. No code, and no contact with that thread — its branches, PR #530 and its darkroom run folder were read only, and nothing was written to any of them.

The one defect, filed first

docs/todo/2026-09-12-ci-installs-no-elephant-so-the-surrogate-tests-skip.mdci.yml installs .[ui,dl], tests/test_surrogates.py opens with importorskip("elephant"), and the surrogates extra pinning elephant==1.2.1 exists only on the draft branch. So the adapter written to correct Elephant's silent failures is exercised on one laptop. ⚠ Closes with that branch landing, not before.

What the paper does and does not license

  • …-stellas-criterion-is-a-significance-test-not-a-training-signal.md — their criterion is the false-positive rate of a significance test; a training contrast is strictly stronger, because a surrogate can be conservative for a test and still hand a discriminator a marginal feature. Their five reasons for preferring trial shifting are properties of a test procedure, and the trial structure it needs does not exist in continuous imaging. Their dead-time practice — per-neuron minimum ISI, capped at 4 ms, because at low rates the minimum stops being biology — independently supports this project's ruling that τ is the producer's to declare.
  • …-count-preservation-after-encoding-is-the-gate-stella-actually-found.md — their mechanism for uniform dither's failure is that binarization deletes events, not that intervals get too short. This encoder binarizes and clips onsets onto the boundary frames, so both channels are present and one is worse here. A per-ROI count of onsets surviving encoding could not be found in surrogate_stats.py by name; establishing whether it exists is the first task.

Reading the run that already happened

  • …-join-the-leak-results-to-the-destruction-results.md — both halves are in the run folder and the join has not been made. Do-nothing survives the leak detector in every stream, which is what proves that detector is a gate and not a score. Most other survivors sit at one frame of displacement, 0.1 s against seconds-scale coordination. Rigid shift is the only candidate surviving out to 1.6 s fast and 1.4 s slow.
  • …-the-verdict-rule-needs-a-displacement-floor.md — derived from destruction, not from test power, which is the defect the second review round already found once in the control gate.
  • …-rigid-shift-without-wrap-loses-onsets-at-the-window-edge.md — truncation at the window end is an event-count difference: Stella's failure mode arriving by another door. The generation margin may already absorb it, and that should be measured rather than assumed.
  • …-joint-isi-dithering-is-unmeasured-not-failed.md — every cell was dropped as intractable. Under a no-pruning ruling, dropping a candidate on resource limits is a decision, and an empty row reads like a negative result.
  • …-the-label-free-detector-thread-has-no-exit-criterion.md — written before the numbers are read, or it cannot tell "not yet" from "no".

Numbers quoted in the todos are aggregates read from the run's run_summary.json and the discriminator metadata on 2026-09-12, each with its provenance stated in the file. No per-recording data and no recording ids.

🤖 Generated with Claude Code

…todos

Tony asked for an independent plan of action on the label-free detector
thread, from a session not working on it. This is that plan, filed where
open work belongs rather than as prose. Nothing here touches the thread:
no writes to its branches, its PR or its run folder, all of which were
read only.

CI installs no Elephant, so the surrogate screen's generator tests skip on
all three Python legs. The extra that would fix it exists only on the draft
branch, so this closes with that branch, not before. Filed first because it
is the one item that is a defect rather than a judgment.

The paper answers a different question than the screen is asking. Stella's
criterion is the false-positive rate of a significance test; a training
contrast is a stronger requirement, because a surrogate can be conservative
for a test and still hand a discriminator a marginal feature. Their reasons
for preferring trial shifting are properties of a test procedure, and the
trial structure it needs does not exist in continuous imaging.

Their diagnosed mechanism for uniform dither is a count difference, not an
interval one: dithering deletes nothing, binarization does. This project's
encoder binarizes and also clips onsets onto the boundary frames, so both
channels are present and one is worse here. Whether a per-ROI count of
onsets surviving encoding is measured at all could not be established from
the statistics module and is the first thing to check.

The run's own leak results already order the candidates, and nobody has
read them against the destruction results, which are on the same disk. Do
nothing survives in every stream, which is what proves the leak filter is
a gate and not a score; most other survivors sit at one frame of
displacement, which is 0.1 s against seconds-scale coordination; and rigid
shift is the only candidate surviving where displacement is comparable to
real event structure. So the verdict rule needs a displacement floor
derived from destruction, and rigid shift needs testing at its weak point,
which is the edge it does not wrap — a count difference again.

Joint-ISI dithering was dropped as intractable at every cell, and an empty
row reads like a negative result. Under a no-pruning ruling that is a
decision, not a resource fact.

And the thread has no exit criterion. If every candidate that does not leak
also fails to destroy coordination, self-supervision is not available on
this data, and that is a result about the preparation worth writing up
rather than a failure. The rule for reading it that way has to be written
before the numbers are.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@syncytium2
syncytium2 merged commit 70e6e6c into main Sep 12, 2026
3 checks passed
@syncytium2
syncytium2 deleted the file-the-surrogate-screen-todos branch September 12, 2026 13:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants