Port OspreySharp pass-2 (transfer-compete, protein-compact frozen) + learned pick, off by default

[maccoss]

Summary

• Ports the OspreySharp pass-2 FDR modes and the learned per-platform peak-pick model, all off by default (a plain run matches the legacy product pick + percolator-retrain golden). OSPREY_PASS2_QVALUE=transfer-compete applies the frozen 1st-pass model over the full pre-compaction population; protein-compact constrains that competition to the >=2-peptide protein stratum. Both stream one file at a time via compute_full_population_fdr_streaming (no retrain). The learned pick model is opt-in via OSPREY_PICK_LDA (OSPREY_PICK_LDA_MODEL still overrides).
• Deterministic shared-boundary charge tie-break (lowest charge) in build_shared_boundaries_from_plan, matching C# BuildSharedBoundaries, so the blib RetentionTimes stay byte-identical cross-impl.

Requested by Mike.

Test plan

• cargo fmt --check / clippy -D warnings / test
• Cross-impl byte-parity with C# at 1e-9 on Stellar 3-file (protein-compact frozen: 45,900 precursors; Stage 7 protein FDR + blib content PASS)

Co-Authored-By: Claude noreply@anthropic.com

[Copilot]

Pull request overview

Ports OspreySharp’s optional “pass-2” FDR strategies (frozen-model transfer-compete and protein-compact) plus an opt-in learned peak-picking model, aiming for cross-implementation byte parity while keeping legacy behavior unchanged unless env-vars are set.

Changes:

• Adds OSPREY_PASS2_QVALUE-controlled second-pass behavior, including a frozen 1st-pass model capture and a streaming full-population target/decoy recompute path.
• Introduces OSPREY_PICK_LDA / OSPREY_PICK_LDA_MODEL learned peak-pick scoring (resolution-keyed defaults), integrated into run_search when enabled.
• Makes shared-boundary selection deterministic on run-q ties by breaking ties on lowest charge to avoid cross-impl ordering drift.

Reviewed changes

Copilot reviewed 5 out of 6 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
crates/osprey/src/pipeline.rs	Implements frozen-model capture + pass-2 recompute wiring, protein-compact stratum construction, deterministic shared-boundary tie-break, and learned pick integration in run_search.
crates/osprey/src/pick_lda.rs	Adds the learned peak-pick model implementation and env-var based selection.
crates/osprey/src/pass2_qvalue.rs	Adds pass-2 mode selection and the streaming full-population FDR recompute helper used by transfer-compete/protein-compact.
crates/osprey/src/lib.rs	Exposes the new pick_lda and pass2_qvalue modules.
crates/osprey/Cargo.toml	Adds osprey-ml dependency for PEP estimation/standardizer support in pass-2 recompute.
Cargo.lock	Locks the new dependency edge for osprey-ml.

Comments suppressed due to low confidence (1)

crates/osprey/src/pipeline.rs:6276

• compute_pass2_transfer_compete uses read_fdr_scores_scalars(...).unwrap_or_default(), which silently turns an unreadable/invalid sidecar into an empty population and can yield bogus q-values/PEPs while still returning true. This should trigger a fallback to the 2nd-pass retrain instead.

    // Stream the full-population competition; only one file's scalars resident at a time.
    let read_file = |file_idx: usize| -> (Vec<u32>, Vec<f64>) {
        read_fdr_scores_scalars(&sidecar_paths[file_idx]).unwrap_or_default()
    };

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[maccoss]

Added a checked_mul/checked_add guard in 85ebc47, so an oversized/corrupt header count now returns None (caller falls back to the retrain) instead of wrapping in release.

On the pass byte: read_fdr_scores_scalars is only ever called with the explicitly-constructed {file}.1st-pass.fdr_scores.bin path (compute_pass2_transfer_compete builds it by name), so a 2nd-pass file cannot be selected here, and the magic + version + exact-size checks reject anything else. The pass-byte validation lives in load_fdr_scores_sidecar, which is the reader that takes an expected_pass.

[maccoss]

Two different reads here; splitting them:

6254 (PIN feature load) — not a correctness risk. frozen is the 1st-pass model that wrote the sidecar scalars, so for a non-rescored survivor frozen.score(features) reproduces its sidecar score exactly — the override is a no-op. The survivors that genuinely need re-scoring are the reconciliation-rescored ones, which take the overlay path (independent of this parquet load). So a failed load cannot corrupt q-values. I added a warn! for observability in 85ebc47 rather than falling back to the retrain.

6276 (full-population scalar sidecar, the suppressed comment) — this one had FDR consequences and is fixed. unwrap_or_default() silently turned a present-but-corrupt sidecar into an empty population, understating the null and making q-values anti-conservative while still returning true. It now flags the read failure and falls back to the 2nd-pass retrain, matching the "missing sidecar -> retrain" contract in the function docstring.

[maccoss]

Fixed in 85ebc47. The "(default)" was stale from when the LDA pick was the default; reworded so the comment states the product form is the default and the learned model is opt-in via OSPREY_PICK_LDA.