Remove pass-2 Percolator, flip the pass-2 and pick defaults, reject I/L-isobaric decoys - #60
Merged
Merged
Conversation
Isoleucine and leucine have identical residue masses, so a decoy that differs from a real target only by I<->L substitutions is precursor-mass-identical and produces an identical b/y ladder. It cannot lose a target/decoy competition against that target on fragment evidence, because mass spectrometry cannot tell them apart, so it is not an honest null. The fragment-overlap gate does not catch this. It compares a candidate to its own source target, while a collision of this kind is an isobaric match to a different target that the gate never looks at. An exact-string collision audit consequently reports zero and misses the population entirely. Simulating this path over a 1,390,979-target Astral set with the overlap gate on gives 0 exact collisions and 742 I/L-isobaric ones (0.0534%), for example AAEESLR -> LSEEAAR. Normalising I to L before the collision check subsumes the exact check rather than adding a second lookup, since a sequence without isoleucine normalises to itself. The new test asserts that the overlap gate PASSES a candidate the collision index rejects, which is what shows the two checks are independent rather than redundant. Also removes the percolator pass-2 mode and moves two defaults, matching the C# implementation so the cross-impl comparison keeps its meaning: - OSPREY_PASS2_QVALUE now defaults to protein-compact. The percolator mode retrained the 2nd-pass SVM over the compacted pool, whose decoys have already been stripped, so the retrained null is thin and the reported q-values are anti-conservative. The 1st-pass linear model is now the model for pass 2 in every mode; only the OSPREY_PROTEIN_COMPACT_RETRAIN diagnostic still retrains. - An unrecognized OSPREY_PASS2_QVALUE now panics instead of silently falling back to a default the caller did not ask for. - OSPREY_PICK_LDA defaults on, with =0 as the opt-out so the two picks stay comparable.
A peak that reconciliation moved, or that gap-fill created, carries a new composite score and no longer has a valid first-pass run q: the old q described a peak that no longer exists, and the post-rescore overlay zeroes it precisely to say so, on the contract that the second pass recomputes it. protein-compact was the only pass-2 mode that never recomputed part of its population. Its off-stratum survivors are meant to ride through on their first-pass q, which is correct for an unchanged peak but meaningless for a changed one - so those peaks kept the q=1 sentinel, which downstream reads as a confident rejection rather than "not yet computed". Changed peaks are now admitted into the same competition the on-stratum members get, and the map-back writes them the q they earned instead of skipping them. Unchanged off-stratum peaks still ride through untouched, so peptides of single-hit proteins stay detectable exactly as the mode intends. The "this peak changed" signal is presence in survivor_score_override, which exists only for peaks re-scored against the reconciled features and is keyed by entry_id - so it means the same thing in-process and on a distributed merge node, where index-keyed sources do not. Admission is by base_id so a target and its paired decoy always enter together; a lone target would auto-win its competition and inflate the null. Mirrors the same fix in Osprey (ProteoWizard/pwiz), and the end-to-end cross-impl gate passes at 1e-9 on Stellar with both sides changed.
The admission used presence in `survivor_score_override` as the "Stage 6 changed" signal, but that map holds every post-reconciliation survivor whose identity resolves in the effective parquet, including files Stage 6 never touched (the effective path falls back to the original parquet). Keying on presence therefore admitted most of the survivor pool and widened the stratum protein-compact exists to enforce. The signal is now a frozen-model score that differs bit-exactly from the entry's 1st-pass sidecar score. An unchanged survivor's reconciled features are its original Stage-4 features and the sidecar score came from those same features under the same averaged model, so the recomputation reproduces it exactly; a moved peak carries rescored features and differs. Both values were already in the loop, and the comparison stays keyed by entry_id so it means the same thing in-process and on a distributed merge node. Mirrors the C# change; the cross-impl gate agrees at rust=29108 cs=29108 delta=0, with Stage 7 and blib content both passing at 1e-9 on Stellar.
protein-compact folded off-stratum changed peaks into the cross-file experiment accumulator while admitting them only in the files that changed them, so their experiment best was a max over that subset while stratum members maxed over all files. That is guaranteed to understate, not merely to skew: reconciliation anchors on the best-scoring peak and corrects the others toward it, so a changed peak never supplied the maximum and a max over changed observations alone can only fall below the true experiment-wide score. Those peptides' q was inflated and they were dropped, breaking the "re-scoping only adds" property. By the same anchor argument reconciliation cannot have invalidated the pass-1 experiment q, so it is carried rather than recomputed. Off-stratum peaks no longer enter the experiment accumulator; only their run-level q is refreshed from the competition they were admitted to. read_fdr_scores_experiment_q reads the pass-1 value from the sidecar, since the post-rescore overlay has already zeroed the in-memory copy. Mirrors the C# change. Cross-impl agrees at rust=29364 cs=29364 delta=0 with Stage 7 passing at 1e-9 on Stellar; an intermediate half-port that removed the accumulator entry without supplying the carry diverged at delta=619, which the gate caught.
brendanx67
added a commit
to ProteoWizard/pwiz-ai
that referenced
this pull request
Aug 4, 2026
* maccoss/osprey#60 merged the same day as fc51d72, so master and main never described different pass-2 behavior * #4515 closed, with the closure naming what it does NOT cover: the set-wise isobaric shadow gate (reverted and shelved, not deferred) and maccoss/Carafe#9, still open * #4484 left open on purpose - it asks which pass-2 method to settle on, and this PR chose a default without claiming it final Co-Authored-By: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Retires the second-pass Percolator retrain, makes
protein-compactand the learned peak pickthe defaults, and rejects I/L-isobaric generated decoys. Mirrors OspreySharp
ProteoWizard/pwiz#4528; the two sides are held
byte-equal by the end-to-end cross-impl comparison, which is the reason for landing them together.
Why pass-2 Percolator goes
It was over optimistic and produced a broken FDP-vs-q curve. Compaction strips most decoys from
the pool the second pass retrains on, so the null is thin and the reported q anti-conservative --
1.57% true FDP at a nominal 1% on Stellar libdecoy entrapment against 0.92% for the first-pass q,
and ~9% on an 82-file SEA-AD set. The linear model trained by the first-pass SVM is now the model
for pass 2 in every mode.
The mode is removed rather than demoted, and an unrecognized
OSPREY_PASS2_QVALUEis now an errorraised at startup rather than at the merge node, so a stale sweep script fails in seconds instead
of after Stage 1-5.
Measured on the entrapment oracle, with pass 1 byte-identical across the change so the whole move
is pass 2: on the Stellar libdecoy set, pass-2 true FDP goes from 1.52% to 0.61% at a nominal 1%,
for a 1.6% cost in accepted precursors, and it still adds 9.9% over pass 1 while now measuring
below the line rather than above it. The falling count is not a regression -- a mode that accepts
more at a nominal 1% while measuring 1.5% true FDP is miscalibrated, not better.
protein-compactis not claimed to be fully valid, nor to be the final pass-2 method. It istarget-conditioned and measured anti-conservative at 82 files, and how on-stratum changed peaks
should get their experiment q is still open. Both flipped defaults stay togglable so the arms
remain comparable.
Two correctness fixes that came out of the flip
The changed-peak admission signal was wrong. Admission keyed on presence in the survivor score
override map, and that map is not a "Stage 6 changed" signal -- it holds every post-reconciliation
survivor whose identity resolves, including files Stage 6 never touched, because the effective
path falls back to the original parquet. It is now keyed on a frozen-model score that differs
bit-exactly from the entry's first-pass sidecar score, which is the same discriminator the per-run
q assignment already uses to separate moved from unchanged peaks, and it needs no new plumbing.
Off-stratum peaks re-maxed the experiment accumulator over a subset. They were folded into the
cross-file experiment accumulator but admitted only in the files that changed them, so their
experiment best maxed over a subset of observations. That is not merely asymmetric, it is
guaranteed to understate: reconciliation anchors on the best-scoring peak and corrects the others
toward it, so a changed peak never supplied the maximum. The result inflated those peptides' q and
dropped them. By the same argument the pass-1 experiment q cannot have been invalidated, so it is
now carried rather than recomputed, and only the run-level q is refreshed.
The I/L decoy gate
I and L have identical residue masses, so a decoy differing from a real target only by I<->L is
precursor-mass-identical and produces an identical b/y ladder -- indistinguishable by mass
spectrometry, and detected wherever its target twin is.
The fragment-overlap gate cannot catch this, and that is the part worth keeping: the overlap gate
compares a candidate to its own source target, while a collision is an isobaric match to a
different one. An exact-string audit therefore reports 0 for every library and misses the
population entirely. Simulating the generation path over the 1,390,979-target Astral set with the
overlap gate on: 0 exact collisions, 742 I/L-isobaric ones (0.0534%), e.g.
AAEESLR -> LSEEAAR.The normalised check subsumes the exact one -- a sequence without isoleucine normalises to itself
-- so it replaces rather than supplements it. Carafe already shipped the same check; this closes
the gap on the generation side.
Validation
cargo fmt --check,cargo clippy -D warnings,cargo test --workspace-- all clean.test_collision_check_rejects_a_decoy_isobaric_to_a_different_targetasserts the independenceproperty -- the overlap gate passes a candidate the collision index rejects -- rather than just
the rejection, because a test of the rejection alone would pass equally well if the two checks
were redundant.
Stellar delta=0, and Astral
rust=117783 cs=117783 delta=0with Stage 7 protein FDR andblib content both matching at 1e-9. Both implementations moved from 119,088 to 117,783 under the
two correctness fixes independently, which is what distinguishes one change mirrored correctly
from two changes that merely resemble each other.
That last point is not decoration: an intermediate state of this branch removed the accumulator
entry without supplying the carry, and the gate immediately reported Rust dropping 619 precursors
against C#. An earlier partial mirror of the admission filter showed up the same way at -152. Both
halves of a pass-2 change have to land together.
Note on scope
OspreySharp additionally puts both flipped defaults into its resume validity key, so a run under
the new defaults cannot adopt cached artifacts written under the old ones. There is no equivalent
change here because this tree has no resume-sidecar or validity-key system.