Skip to content

Remove pass-2 Percolator, flip the pass-2 and pick defaults, reject I/L-isobaric decoys - #60

Merged
brendanx67 merged 4 commits into
mainfrom
feature/decoy-il-gate-and-default-flip
Aug 4, 2026
Merged

Remove pass-2 Percolator, flip the pass-2 and pick defaults, reject I/L-isobaric decoys#60
brendanx67 merged 4 commits into
mainfrom
feature/decoy-il-gate-and-default-flip

Conversation

@brendanx67

Copy link
Copy Markdown
Collaborator

Retires the second-pass Percolator retrain, makes protein-compact and the learned peak pick
the defaults, and rejects I/L-isobaric generated decoys. Mirrors OspreySharp
ProteoWizard/pwiz#4528; the two sides are held
byte-equal by the end-to-end cross-impl comparison, which is the reason for landing them together.

Why pass-2 Percolator goes

It was over optimistic and produced a broken FDP-vs-q curve. Compaction strips most decoys from
the pool the second pass retrains on, so the null is thin and the reported q anti-conservative --
1.57% true FDP at a nominal 1% on Stellar libdecoy entrapment against 0.92% for the first-pass q,
and ~9% on an 82-file SEA-AD set. The linear model trained by the first-pass SVM is now the model
for pass 2 in every mode.

The mode is removed rather than demoted, and an unrecognized OSPREY_PASS2_QVALUE is now an error
raised at startup rather than at the merge node, so a stale sweep script fails in seconds instead
of after Stage 1-5.

Measured on the entrapment oracle, with pass 1 byte-identical across the change so the whole move
is pass 2: on the Stellar libdecoy set, pass-2 true FDP goes from 1.52% to 0.61% at a nominal 1%,
for a 1.6% cost in accepted precursors, and it still adds 9.9% over pass 1 while now measuring
below the line rather than above it. The falling count is not a regression -- a mode that accepts
more at a nominal 1% while measuring 1.5% true FDP is miscalibrated, not better.

protein-compact is not claimed to be fully valid, nor to be the final pass-2 method. It is
target-conditioned and measured anti-conservative at 82 files, and how on-stratum changed peaks
should get their experiment q is still open. Both flipped defaults stay togglable so the arms
remain comparable.

Two correctness fixes that came out of the flip

The changed-peak admission signal was wrong. Admission keyed on presence in the survivor score
override map, and that map is not a "Stage 6 changed" signal -- it holds every post-reconciliation
survivor whose identity resolves, including files Stage 6 never touched, because the effective
path falls back to the original parquet. It is now keyed on a frozen-model score that differs
bit-exactly from the entry's first-pass sidecar score, which is the same discriminator the per-run
q assignment already uses to separate moved from unchanged peaks, and it needs no new plumbing.

Off-stratum peaks re-maxed the experiment accumulator over a subset. They were folded into the
cross-file experiment accumulator but admitted only in the files that changed them, so their
experiment best maxed over a subset of observations. That is not merely asymmetric, it is
guaranteed to understate: reconciliation anchors on the best-scoring peak and corrects the others
toward it, so a changed peak never supplied the maximum. The result inflated those peptides' q and
dropped them. By the same argument the pass-1 experiment q cannot have been invalidated, so it is
now carried rather than recomputed, and only the run-level q is refreshed.

The I/L decoy gate

I and L have identical residue masses, so a decoy differing from a real target only by I<->L is
precursor-mass-identical and produces an identical b/y ladder -- indistinguishable by mass
spectrometry, and detected wherever its target twin is.

The fragment-overlap gate cannot catch this, and that is the part worth keeping: the overlap gate
compares a candidate to its own source target, while a collision is an isobaric match to a
different one. An exact-string audit therefore reports 0 for every library and misses the
population entirely. Simulating the generation path over the 1,390,979-target Astral set with the
overlap gate on: 0 exact collisions, 742 I/L-isobaric ones (0.0534%), e.g. AAEESLR -> LSEEAAR.

The normalised check subsumes the exact one -- a sequence without isoleucine normalises to itself
-- so it replaces rather than supplements it. Carafe already shipped the same check; this closes
the gap on the generation side.

Validation

  • cargo fmt --check, cargo clippy -D warnings, cargo test --workspace -- all clean.
  • test_collision_check_rejects_a_decoy_isobaric_to_a_different_target asserts the independence
    property -- the overlap gate passes a candidate the collision index rejects -- rather than just
    the rejection, because a test of the rejection alone would pass equally well if the two checks
    were redundant.
  • End-to-end cross-impl against OspreySharp, both straight-through, rebuilt binaries:
    Stellar delta=0, and Astral rust=117783 cs=117783 delta=0 with Stage 7 protein FDR and
    blib content both matching at 1e-9. Both implementations moved from 119,088 to 117,783 under the
    two correctness fixes independently, which is what distinguishes one change mirrored correctly
    from two changes that merely resemble each other.

That last point is not decoration: an intermediate state of this branch removed the accumulator
entry without supplying the carry, and the gate immediately reported Rust dropping 619 precursors
against C#. An earlier partial mirror of the admission filter showed up the same way at -152. Both
halves of a pass-2 change have to land together.

Note on scope

OspreySharp additionally puts both flipped defaults into its resume validity key, so a run under
the new defaults cannot adopt cached artifacts written under the old ones. There is no equivalent
change here because this tree has no resume-sidecar or validity-key system.

Isoleucine and leucine have identical residue masses, so a decoy that differs
from a real target only by I<->L substitutions is precursor-mass-identical and
produces an identical b/y ladder. It cannot lose a target/decoy competition
against that target on fragment evidence, because mass spectrometry cannot tell
them apart, so it is not an honest null.

The fragment-overlap gate does not catch this. It compares a candidate to its
own source target, while a collision of this kind is an isobaric match to a
different target that the gate never looks at. An exact-string collision audit
consequently reports zero and misses the population entirely. Simulating this
path over a 1,390,979-target Astral set with the overlap gate on gives 0 exact
collisions and 742 I/L-isobaric ones (0.0534%), for example AAEESLR -> LSEEAAR.

Normalising I to L before the collision check subsumes the exact check rather
than adding a second lookup, since a sequence without isoleucine normalises to
itself. The new test asserts that the overlap gate PASSES a candidate the
collision index rejects, which is what shows the two checks are independent
rather than redundant.

Also removes the percolator pass-2 mode and moves two defaults, matching the
C# implementation so the cross-impl comparison keeps its meaning:

- OSPREY_PASS2_QVALUE now defaults to protein-compact. The percolator mode
  retrained the 2nd-pass SVM over the compacted pool, whose decoys have already
  been stripped, so the retrained null is thin and the reported q-values are
  anti-conservative. The 1st-pass linear model is now the model for pass 2 in
  every mode; only the OSPREY_PROTEIN_COMPACT_RETRAIN diagnostic still retrains.
- An unrecognized OSPREY_PASS2_QVALUE now panics instead of silently falling
  back to a default the caller did not ask for.
- OSPREY_PICK_LDA defaults on, with =0 as the opt-out so the two picks stay
  comparable.
A peak that reconciliation moved, or that gap-fill created, carries a new
composite score and no longer has a valid first-pass run q: the old q described
a peak that no longer exists, and the post-rescore overlay zeroes it precisely
to say so, on the contract that the second pass recomputes it.

protein-compact was the only pass-2 mode that never recomputed part of its
population. Its off-stratum survivors are meant to ride through on their
first-pass q, which is correct for an unchanged peak but meaningless for a
changed one - so those peaks kept the q=1 sentinel, which downstream reads as a
confident rejection rather than "not yet computed".

Changed peaks are now admitted into the same competition the on-stratum members
get, and the map-back writes them the q they earned instead of skipping them.
Unchanged off-stratum peaks still ride through untouched, so peptides of
single-hit proteins stay detectable exactly as the mode intends.

The "this peak changed" signal is presence in survivor_score_override, which
exists only for peaks re-scored against the reconciled features and is keyed by
entry_id - so it means the same thing in-process and on a distributed merge
node, where index-keyed sources do not. Admission is by base_id so a target and
its paired decoy always enter together; a lone target would auto-win its
competition and inflate the null.

Mirrors the same fix in Osprey (ProteoWizard/pwiz), and the end-to-end
cross-impl gate passes at 1e-9 on Stellar with both sides changed.
The admission used presence in `survivor_score_override` as the "Stage 6
changed" signal, but that map holds every post-reconciliation survivor whose
identity resolves in the effective parquet, including files Stage 6 never
touched (the effective path falls back to the original parquet). Keying on
presence therefore admitted most of the survivor pool and widened the stratum
protein-compact exists to enforce.

The signal is now a frozen-model score that differs bit-exactly from the
entry's 1st-pass sidecar score. An unchanged survivor's reconciled features are
its original Stage-4 features and the sidecar score came from those same
features under the same averaged model, so the recomputation reproduces it
exactly; a moved peak carries rescored features and differs. Both values were
already in the loop, and the comparison stays keyed by entry_id so it means the
same thing in-process and on a distributed merge node.

Mirrors the C# change; the cross-impl gate agrees at rust=29108 cs=29108
delta=0, with Stage 7 and blib content both passing at 1e-9 on Stellar.
protein-compact folded off-stratum changed peaks into the cross-file experiment
accumulator while admitting them only in the files that changed them, so their
experiment best was a max over that subset while stratum members maxed over all
files.

That is guaranteed to understate, not merely to skew: reconciliation anchors on
the best-scoring peak and corrects the others toward it, so a changed peak never
supplied the maximum and a max over changed observations alone can only fall
below the true experiment-wide score. Those peptides' q was inflated and they
were dropped, breaking the "re-scoping only adds" property.

By the same anchor argument reconciliation cannot have invalidated the pass-1
experiment q, so it is carried rather than recomputed. Off-stratum peaks no
longer enter the experiment accumulator; only their run-level q is refreshed
from the competition they were admitted to. read_fdr_scores_experiment_q reads
the pass-1 value from the sidecar, since the post-rescore overlay has already
zeroed the in-memory copy.

Mirrors the C# change. Cross-impl agrees at rust=29364 cs=29364 delta=0 with
Stage 7 passing at 1e-9 on Stellar; an intermediate half-port that removed the
accumulator entry without supplying the carry diverged at delta=619, which the
gate caught.
@brendanx67
brendanx67 merged commit fc51d72 into main Aug 4, 2026
3 checks passed
@brendanx67
brendanx67 deleted the feature/decoy-il-gate-and-default-flip branch August 4, 2026 09:57
brendanx67 added a commit to ProteoWizard/pwiz-ai that referenced this pull request Aug 4, 2026
* maccoss/osprey#60 merged the same day as fc51d72, so master and main never
  described different pass-2 behavior
* #4515 closed, with the closure naming what it does NOT cover: the set-wise
  isobaric shadow gate (reverted and shelved, not deferred) and maccoss/Carafe#9,
  still open
* #4484 left open on purpose - it asks which pass-2 method to settle on, and
  this PR chose a default without claiming it final

Co-Authored-By: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant