Skip to content

feat: honour the model's own silence, and spend it on recall - #84

Merged
Endika merged 6 commits into
mainfrom
feat/silence-gate
Aug 27, 2026
Merged

feat: honour the model's own silence, and spend it on recall#84
Endika merged 6 commits into
mainfrom
feat/silence-gate

Conversation

@Endika

@Endika Endika commented Aug 27, 2026

Copy link
Copy Markdown
Owner

The vocabulary model has carried a trained __NADA__ class — "nobody is signing", cut by
health_dataset.py from the gaps between LSE-Health's translated sentences — since the
co-articulated retrain. Nothing on this side read it: classify treated it as an ordinary
concept and createGloss humanised it to the word __nada__, so the app wrote 131 of 1,477
words as __nada__
on held-out signers, and health_words.py counted every one of those as a
word written. Over half the pause-babble figure this repo published was the model's own silence.

Nothing is retrained here and lse-vocabulary.bin is byte-identical.

What it buys

Four seeds, floor 850, four held-out signers, 1,060 annotated instances and 211 windows landing
in gaps between annotated sentences.

before after
signs written correctly, continuous signing 37.3% (sd 0.9) 44.2% (sd 0.7)
words written into pauses 30.0% (sd 2.4) 19.9% (sd 1.9)
__nada__ reaching the transcript 131 of 1,477 0

Both axes improve at once, which the floor/gate grid had never managed. The recall gain holds in
every one of the four seeds, so it is not the best of four.

The two moves are separate and in that order:

  1. Honour the abstention. If the model's silence class wins, nothing is written — not even
    the runner-up, which would contradict the answer the model just gave. Measured, this alone
    takes pause babble from 27.5% to 10.9% on shipped weights with recall unmoved at 38.1%. A
    filtering policy that writes the runner-up scored identically to the word, because under an
    abstention no real concept ever cleared the gate; the abstention is what ships and a test
    pins the difference.
  2. Spend the freed room on recall. With babble no longer counting the abstention, the window
    gate comes down from 0.60 to 0.45.

Notes for review

  • The class is declared in the manifest as abstentionConcept, not hardcoded in TypeScript, and
    check_manifest.py now fails on any __-prefixed concept that goes undeclared. This bug was
    two places having to agree with nothing checking, which is the shape of the handedness and
    normalisation bugs already recorded in tools/train/README.md.
  • The engine's own floor drops to 0.30 so it stays below the gate. Otherwise both floors sit at
    0.45, the stabilizer veto becomes unreachable and a real diagnostic dies. The effective
    threshold a word must clear is 0.45 either way, which is exactly what the bench measured.
  • abstention is a fourth WindowVeto. The model answering "nobody is signing" is an answer,
    not a threshold, and reporting it as classifier sends anyone reading the panel after a floor
    that was never involved.
  • The About panel said 287 signos when one of those 287 is the abstention. Now 286, and
    check_manifest.py subtracts the abstention so it cannot drift back.

What this does not do

No learned abstention for the remaining babble, and tools/train/README.md says why with the
measurement: the class is precise (argmax over a real sign 0.9%, never above 0.60) but of the
pauses where the app was writing a real word, P(__NADA__) > 0.30 in 0.0% of them. Abstention
and remaining babble are disjoint. The negatives that would fix the rest are not labellable in
this corpus — 101 annotated sign types against the 90-150 signs a minute of fluent discourse — and
with 23 of 211 windows left and ±6 points of sampling error, a further 3 or 4 points could not be
demonstrated here even if it worked.

No segmenter changes: floor, grace, window boundaries and DominantHandTracker are untouched.

Verified: npm run lint, npm run typecheck, npm test (205 tests, 11 new), the 33 Python bench
tests, check_manifest.py, and npm run build.

@Endika
Endika merged commit bf4a35c into main Aug 27, 2026
8 checks passed
@Endika
Endika deleted the feat/silence-gate branch August 27, 2026 10:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant