Skip to content

A trained model outlives its process, so the learned half of the loop closes too - #508

Merged
syncytium2 merged 1 commit into
mainfrom
weights-survive-the-process
Sep 9, 2026
Merged

A trained model outlives its process, so the learned half of the loop closes too#508
syncytium2 merged 1 commit into
mainfrom
weights-survive-the-process

Conversation

@syncytium2

Copy link
Copy Markdown
Owner

The settings half landed in #507; this is the other one. docs/pipeline.md's blocker list
is now empty
— both items that sat upstream of everything else on that page are closed.

  • bugarach.learn.checkpoint saves and loads a trained model.
  • run_learned_on_folder.py --save-models writes them.
  • bugarach detect --model ckpt.json runs one over an export folder, repeatable, beside
    the six.

Verified the only way that counts

On the pilot APV+CNQX+GZ folder: train two models and save them, then detect with those files
from a separate process.

model / stream in-process from file
tube / fast 101 101
tube / slow 253 253
tube_guard / fast 112 112
tube_guard / slow 257 257

Identical call for call. 31 KB per checkpoint.

JSON, not torch.save

These nets are 1,149–2,393 parameters — smallness is their claimed advantage — so a
checkpoint is tens of kilobytes of numbers and can be a format a person reads, a diff shows
and JSON.parse loads. That is what makes a model shareable in both directions rather
than only out of one.

And torch.save is pickle. ADR-0005's target flow is a user downloading a folder of models
from the site; a format where opening a stranger's model executes their code cannot be that
format. Nothing here needs it — the state dict is arrays of floats.

What travels with the weights

The architecture name and the config it was built with, so moving a registry default
cannot rebuild a different network under old weights. The encoding contractdt, the
onset field, the busiest-first row rule — because a model is a function of the raster it was
shown, and encode's own docstring is emphatic that row order is a coordinate rather than a
label. The operating point, since a model without its threshold is not a detector. And
the provenance, including the torch thread count, because train.THREADS records
reduction order moving F1 by 0.018 on this very code.

What it refuses

A tensor whose shape no longer matches, with both shapes named — that is the failure a
saved model actually has, the file outliving an edit to the architecture, and a half-loaded
model that then scores is worse than one that will not open. A changed encoding contract, for
the same reason. A model whose name collides with one of the six, because detections.csv's
detector column is one namespace across both families. And a bad checkpoint fails on
recording 0, not on 84 of 85.

peek reads provenance without importing torch, so a listing need not instantiate a
network to say what a model is.

Learned calls land in the six ports' contract and are scored inside each analysis window,
the way the flat three are, so no model gets context across a drug transition the others were
denied. Being indistinguishable from a hand-written call in the file is deliberate — which is
why run.json now carries the roster of models that ran. Their n_roi stays NA: the model
emits a per-frame score and never says which cells it meant.

Full suite 2260 passed, 33 skipped, 1 xfailed. Sapper clear.

🤖 Generated with Claude Code

… closes too

Tony: "that is the whole point of the pipeline". The settings half landed in #507; this is
the other one. `docs/pipeline.md`'s blocker list is now empty -- both items that sat upstream
of everything else on that page are closed.

`bugarach.learn.checkpoint` saves and loads a trained model. `run_learned_on_folder.py
--save-models` writes them. `bugarach detect --model ckpt.json` runs one over an export
folder, repeatable, beside the six.

VERIFIED THE ONLY WAY THAT COUNTS, on the pilot APV+CNQX+GZ folder: train two models and
save them, then detect with those files from a SEPARATE process. 101 and 253 calls for tube,
112 and 257 for tube_guard, fast and slow -- identical to the training process, call for
call. 31 KB per checkpoint.

JSON, NOT torch.save, and the second reason settles it. These nets are 1,149 to 2,393
parameters -- smallness is their claimed advantage -- so a checkpoint is tens of kilobytes of
numbers and can be a format a person reads, a diff shows and JSON.parse loads, which is what
makes a model shareable in BOTH directions rather than only out of one. And torch.save is
pickle: ADR-0005's target flow is a user downloading a folder of models from the site, and a
format where opening a stranger's model executes their code cannot be that format. Nothing
here needs it; the state dict is arrays of floats.

WHAT TRAVELS WITH THE WEIGHTS, each because its absence fails silently. The architecture name
AND the config it was built with, so moving a registry default cannot rebuild a different
network under old weights. The encoding contract -- dt, the onset field, the busiest-first row
rule -- because a model is a function of the raster it was shown, and encode's own docstring
is emphatic that row order is a coordinate rather than a label. The operating point, since a
model without its threshold is not a detector. And the provenance, including the torch thread
count, because train.THREADS records reduction order moving F1 by 0.018 on this very code.

WHAT IT REFUSES. A tensor whose shape no longer matches is refused with BOTH shapes named --
that is the failure a saved model actually has, the file outliving an edit to the
architecture, and a half-loaded model that then scores is worse than one that will not open.
A changed encoding contract is refused for the same reason. A model whose name collides with
one of the six is refused because detections.csv's detector column is one namespace across
both families. And a bad checkpoint fails on recording 0, not on 84 of 85.

`peek` reads provenance without importing torch, so a report tool or a listing need not
instantiate a network to say what a model is.

Learned calls land in the six ports' contract and are scored INSIDE each analysis window, the
way the flat three are, so no model gets context across a drug transition the others were
denied. Being indistinguishable from a hand-written call in the file is deliberate, which is
why run.json now carries the roster of models that ran. Their n_roi stays NA: the model emits
a per-frame score and never says which cells it meant.

Full suite 2260 passed, 33 skipped, 1 xfailed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@syncytium2
syncytium2 merged commit 3bb3899 into main Sep 9, 2026
3 checks passed
@syncytium2
syncytium2 deleted the weights-survive-the-process branch September 9, 2026 04:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants