Remove the b<->y intensity swap from generated decoys - #58
Merged
Conversation
Decoy fragments now keep the target's ion type and ordinal (a target y7 yields a
decoy y7); only the m/z is recomputed for the permuted sequence, so the copied
relative intensity stays on the same-numbered ion.
The previous mapping sent target b_k to decoy y_{n-k} and carried the intensity
along with the relabel. The residue-coverage reasoning behind it was sound, but
fragment intensity is dominated by ion TYPE, not by which residues an ion spans:
y ions are systematically more intense than b ions, so the swap inverted the decoy
spectrum's intensity structure relative to any real peptide.
Measured with entrapment (FDRBench-equivalent, pass 1, experiment-wide), true FDP
at a claimed 1% q:
Stellar 10.9% -> 1.5%
Astral 7.6% -> 2.0% (library-decoy reference 1.9%)
On an 82-file SEA-AD set the fixed construction holds at 2.1% and recovers 19% MORE
identifications than Carafe library decoys at matched TRUE 1% FDP.
Skyline, OpenSWATH, DIA-NN, EncyclopeDIA and SpectraST all map the intensity to the
same ion; none of them swaps.
Matches the C# change on ProteoWizard/pwiz branch
Skyline/work/20260725_osprey_gendecoy_decision.
Adds the similarity gate every other implementation surveyed already has (EncyclopeDIA getSmartDecoy 0.4, SpectraST spectral similarity 0.7, OpenSWATH sequence identity 0.5): a candidate decoy whose theoretical b/y ladder coincides with its target's by more than 40% is rejected, and the existing cycling fallback supplies another candidate. Effect at library scale is nil -- on the order of 1e-4 of peptides excluded, with entrapment FDP unchanged within noise. It is kept for robustness at small library scale, where palindromes and low-complexity runs are a far larger fraction and a near-identical decoy would be both real and confusing. The match window is a fixed 0.02 Da rather than the run's fragment tolerance, so the decoy set stays a pure function of the library instead of varying with instrument resolution. The ladder ignores modifications, which shift both sides alike. Both choices keep this rule byte-identical to the C# implementation. Matches ProteoWizard/pwiz branch Skyline/work/20260725_osprey_gendecoy_decision.
…esidue theoretical_ladder derived y ions as (total - prefix), so a single residue missing from the standard mass table made `total` NaN and killed EVERY y ion - not just the ones actually spanning it, which is what the surrounding comment promised. A LEADING unknown residue emptied the ladder outright, and an empty candidate ladder is read by is_candidate_acceptable as "accept", silently switching the similarity gate off for exactly the peptides it could not evaluate. y ions now come from suffix sums, mirroring the prefix sums used for b ions, so only ions that actually span the unknown residue are dropped. This is not hypothetical. The Astral reference library (1,148,928 distinct peptides) contains 60 selenocysteine (U) peptides. With half their ladder missing, the inflated overlap ratio pushed the plain reversal past the 0.4 threshold and forced the cycling fallback for three of them: GPSPPPMAGGUGR was PSPPPMAGGUGGR now GUGGAMPPPSPGR GFVCIVTNVASQUGK was FVCIVTNVASQUGGK now GUQSAVNTVICVFGK SYSSKPPLWAAELCVTUSS was YSSKPPLWAAELCVTUSSS now SUTVCLEAAWLPPKSSYSS Selenocysteine and the ambiguity codes B/Z/X/J/O are all absent from the residue table and do occur in UniProt-derived libraries. Also adds the first tests for decoy construction - the b<->y swap removal and the overlap gate had none - and restores the doc comment for generate_all_with_collision_detection, which the previous commit left attached to MAX_FRAGMENT_OVERLAP. The docs' ion-swap section is rewritten to describe what the code now does, plus a section on the overlap gate. The four tests mirror OspreySharp's DecoyConstructionTest one for one, including the selenocysteine ladder rung counts, so a change on either side that is not mirrored breaks both suites. Matches the C# change on ProteoWizard/pwiz branch Skyline/work/20260726_osprey_regression_redesign.
is_candidate_acceptable compares the FULL theoretical ladder, matching EncyclopeDIA's
rule so that their published 0.4 threshold is measured over their statistic. Two rungs of
that ladder are invariant under any C-terminus-preserving permutation - y1, and b_{n-1},
whose prefix multiset never changes - so they always match and put a structural 1/(n-1)
floor under the overlap ratio. At length 3 that floor is 0.5 against a 0.4 budget: every
candidate would be rejected and the peptide would vanish from the search with no
diagnostic.
Rather than weaken the statistic to defend against input that real libraries do not
produce, the bound is enforced at load: validate_peptide_length rejects a peptide shorter
than MIN_PEPTIDE_LENGTH = 6, naming the offending peptide. At the minimum the floor is
1/5 = 0.2, comfortably clear.
Called by all three format loaders (DIA-NN TSV, blib, elib). An invariant enforced on one
of the paths into the library is not an invariant, and the decoy generator cannot tell
which loader produced its input.
The bound cannot fire on real input: MaxQuant's convention is 6, the Carafe libraries the
reference datasets use start at 7, and the Astral library's shortest peptide is 7. It
exists so that a malformed library fails loudly instead of quietly changing what gets
searched.
Matches the C# change on ProteoWizard/pwiz branch
Skyline/work/20260726_osprey_regression_redesign.
7 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Generated (reverse) decoys were badly miscalibrated: measured against an FDRBench
entrapment oracle they carried roughly 12x the reported error rate, ~11.8% true FDP
at a claimed 1% q on Stellar and 7.6% on Astral. This traces to a single defect.
RecalculateFragmentsmapped target b_k to decoy y_{n-k} and carried the copiedrelative intensity along with the relabel. The reasoning behind it -- "which decoy
ion covers the same residues?" -- was sound, but fragment intensity is dominated by
ion TYPE, not by residue coverage: y ions are systematically more intense than b
ions, so relocating intensity across the swap inverts the strongest structural
feature of the spectrum. A source review of six open-source implementations found
five of six map intensity to the same ion label; this swap was unique to Osprey.
Dropping it recovers a near-honest null with no predictive model:
For scale, Carafe-predicted library decoys measure 1.92% on the same Astral cell, so
the fixed generated decoys reach parity with model-predicted ones. On an 82-file
SEA-AD set the fixed construction recovers 19% MORE real identifications than library
decoys at a matched TRUE 1% FDP, so this is not a sensitivity trade.
The paired decoy-win fraction tells the same story from a decoy-independent angle:
within a target/decoy pair an honest decoy should beat a known-false entrapment
target about half the time. Before the fix it won 0.205 of the time; after, 0.473,
against 0.501 for library decoys.
The second commit adds the similarity gate every surveyed implementation already
carries (EncyclopeDIA 0.4 fragment overlap, SpectraST 0.7 spectral similarity,
OpenSWATH 0.5 sequence identity). Its effect at library scale is nil -- ~1e-4 of
peptides excluded, entrapment FDP unchanged within noise -- and it is kept for
robustness at small library scale, where palindromes and low-complexity runs are a
far larger fraction.
Two things measured and REJECTED, recorded so they are not re-litigated: Skyline's
+10-unit precursor m/z shift (net negative here, 1.47 -> 1.96% Stellar and
2.03 -> 3.49% Astral -- it buys MS1 separation, but a decoy-only handicap is
spurious separation), and both similarity gates as an FDP lever.
Cross-implementation parity, both sides built from matching source:
Stage 7 protein FDR and full blib content both compare clean per-column at 1e-9.
The matching ProteoWizard/pwiz change is on branch
Skyline/work/20260726_osprey_regression_redesign, which also re-records the C#regression goldens against this construction and adds a libdecoy+entrapment
regression dataset with FDR calibration spot checks. The two must land together:
the C# golden re-recording is only defensible alongside this change.
cargo fmt, clippy -D warnings and cargo test all pass.
Added 2026-07-27: the ladder had a second defect, and the invariant it relies on was enforced on one path
theoretical_ladderdropped every y ion when any residue was unknown. It derived y astotal - prefix, so one residue missing from the mass table madetotalNaN and killedthe whole y series - not just the ions spanning it, which is what the comment above it
promised. A LEADING unknown emptied the ladder outright, and an empty candidate ladder is
read by
is_candidate_acceptableas "accept": the gate silently switched itself off forexactly the peptides it could not evaluate.
Not hypothetical. The Astral reference library (1,148,928 distinct peptides) carries 60
selenocysteine (U) peptides; the Stellar library carries none. With half their ladder
missing, the inflated overlap ratio pushed the plain reversal past 0.4 and forced the
cycling fallback for three of them:
y ions now come from suffix sums, mirroring the prefix sums used for b ions.
MIN_PEPTIDE_LENGTH = 6is now enforced by all three format loaders (DIA-NN TSV,blib, elib) rather than assumed. The overlap gate compares the FULL ladder, so its two
permutation-invariant rungs put a structural
1/(n-1)floor under the ratio; at length 3that floor is 0.5 against a 0.4 budget and every candidate would be rejected, dropping the
peptide from the search with no diagnostic. At the enforced minimum it is 0.2. The bound
cannot fire on real input - the Astral library's shortest peptide is 7 residues - so it
exists to make a malformed library fail loudly instead of quietly changing what is
searched.
Tests. Neither the swap removal nor the overlap gate had any. Four were added, mirroring
OspreySharp's
DecoyConstructionTestone for one: ion type + ordinal preserved with the m/zread off the decoy's own ladder; the gate rejects an isobaric near-duplicate (
AILLAK->ALLIAK, where I/L isobaricity makes every rung coincide) and accepts an ordinarytryptic reversal, so a gate that rejected everything cannot pass; generation never emits a
rejected candidate; and the ladder drops only the ions spanning an unknown residue. Both
implementations assert the same rung counts, so a change on either side that is not mirrored
breaks both suites.
Also restores the doc comment for
generate_all_with_collision_detection, which theoverlap-gate commit left attached to
MAX_FRAGMENT_OVERLAP, and rewrites the docs'ion-swap section to describe what the code now does.
The matching OspreySharp PR is ProteoWizard/pwiz#4480 and must land with this one. It
re-records the Astral golden for the corrected ladder and adds the regression coverage that
would have caught the swap.
Gates:
cargo fmt --check,clippy -D warnings,cargo testall green.