From ddc45683f2dfa6972c06798f6a69ecf917f5486d Mon Sep 17 00:00:00 2001 From: Neil Singh Date: Wed, 19 Aug 2026 21:46:11 -0400 Subject: [PATCH] Add ranking score pipeline doc Documents the four post-Phoenix scoring stages (base weighted score, author cold-start, author diversity decay, out-of-network discount) with formulas, default param values, and worked examples, plus a note on why slate_context intentionally stays stale for cached requests. --- docs/RANKING_SCORE_PIPELINE.md | 213 +++++++++++++++++++++++++++++++++ 1 file changed, 213 insertions(+) create mode 100644 docs/RANKING_SCORE_PIPELINE.md diff --git a/docs/RANKING_SCORE_PIPELINE.md b/docs/RANKING_SCORE_PIPELINE.md new file mode 100644 index 00000000..8ec1d35a --- /dev/null +++ b/docs/RANKING_SCORE_PIPELINE.md @@ -0,0 +1,213 @@ +# From Phoenix probabilities to a final score: the ranking pipeline in detail + +The [README's Scoring and Ranking section](../README.md#scoring-and-ranking) describes the four +stages a candidate goes through after Phoenix predicts its action probabilities: the base +weighted score, author cold-start, author diversity, and the out-of-network discount. This doc +fills in the formulas, default values, and code references behind each stage, with worked +examples, so you don't have to reconstruct them from +[`home-mixer/scorers/ranking_scorer.rs`](../home-mixer/scorers/ranking_scorer.rs) and +[`home-mixer/scorers/author_cold_start.rs`](../home-mixer/scorers/author_cold_start.rs) yourself. + +All four stages run in that order, each one operating on the output of the last: + +``` +base weighted score → author cold-start boost → author diversity decay → OON discount +``` + +## Stage 1: the base weighted score + +Phoenix outputs a probability (or, for a few heads like dwell time, a continuous value) per +action per candidate. `RankingScorer` turns those into one number per candidate in three steps. + +**`apply`** ([`ranking_scorer.rs:447`](../home-mixer/scorers/ranking_scorer.rs#L447)) is the +atomic operation — it multiplies one action's probability by its configured weight: + +```rust +fn apply(score: Option, weight: f64) -> f64 { + score.unwrap_or(0.0) * weight +} +``` + +**`compute_weighted_parts`** ([`ranking_scorer.rs:460`](../home-mixer/scorers/ranking_scorer.rs#L460)) +calls `apply` for each of the ~26 action terms (favorite, reply, retweet, report, block, mute, +dwell, etc.) using the weights in +[`home-mixer/params/param.rs`](../home-mixer/params/param.rs), then splits the results into two +running totals: `pos`, the sum of every non-negative term, and `neg`, the summed magnitude of +every negative term (not-interested, block, mute, report, not-dwelled all carry negative +weights). + +**`offset_score`** ([`ranking_scorer.rs:554`](../home-mixer/scorers/ranking_scorer.rs#L554)) takes +`combined_score = pos - neg` and remaps it: + +```rust +pub(crate) fn offset_score(combined_score: f64, w: &ScoringWeights) -> f64 { + if w.total_sum == 0.0 { + combined_score.max(0.0) + } else if combined_score < 0.0 { + (combined_score + w.negative_sum) / w.total_sum * NEGATIVE_SCORES_OFFSET + } else { + combined_score + NEGATIVE_SCORES_OFFSET + } +} +``` + +`NEGATIVE_SCORES_OFFSET` is `0.001` +([`home-mixer/params/config.rs:40`](../home-mixer/params/config.rs#L40)). Net-negative candidates +get squeezed into `[0, 0.001)`; net-non-negative candidates get shifted up by a flat `0.001`. The +effect: every candidate whose predicted actions are net-good outranks every candidate whose +predicted actions are net-bad, while relative order is preserved within each group. + +**Worked example.** Using the default weights (`FavoriteWeight = 0.5`, +`ReplyWeight = 5.0`, `ReportWeight = -234.0`) and a candidate with +`P(favorite) = 0.10`, `P(reply) = 0.02`, `P(report) = 0.001`: + +``` +favorite term = 0.10 × 0.5 = 0.050 +reply term = 0.02 × 5.0 = 0.100 +report term = 0.001 × -234 = -0.234 + +pos = 0.150, neg = 0.234 +combined_score = 0.150 - 0.234 = -0.084 (net negative) +``` + +That -234 weight looks disproportionate until you remember it's multiplying a *probability*, not +a raw count — a 0.1% predicted report chance still outweighs a 10% predicted favorite chance, +which is the intended behavior: a small chance of a severe negative action should dominate a +larger chance of a mild positive one. + +## Stage 2: author cold-start + +Runs in [`author_cold_start.rs`](../home-mixer/scorers/author_cold_start.rs), gated by +`EnableViewerColdStart`. New/small accounts get almost no engagement signal, so Phoenix's +probability estimates for their posts are naturally weak — which keeps them from ever being +shown, which keeps them from ever collecting signal. This stage breaks that loop by forcing one +eligible candidate per request into a specific visible slot. + +**Eligibility** (`cold_start_base_eligible`, +[`author_cold_start.rs:138`](../home-mixer/scorers/author_cold_start.rs#L138), plus filters in +`apply_cold_start`, [`author_cold_start.rs:257`](../home-mixer/scorers/author_cold_start.rs#L257)): +an original post (not a reply/retweet), from an author with at most `ColdStartFollowerCap` +(default `1000`) followers, with fewer than `ColdStartImpressionThreshold` (default `1000`) views +on this specific post, fresh enough (`ColdStartMaxPostAgeSecs`), and already ranked within the +top `LowImpressionsMaxPositionRatio` fraction of the slate. + +**Picking one** (`pick_thompson` / `sample_reward`, +[`author_cold_start.rs:215`](../home-mixer/scorers/author_cold_start.rs#L215)): when +`EnableColdStartThompsonSampling` is on, each eligible candidate's true engagement rate is +modeled as `Beta(alpha0 + favs, beta0 + (views - favs))` and sampled from — a standard Bayesian +bandit that lets a post with zero engagement history still occasionally win, rather than always +being assumed bad because it has no data yet. + +**The boost** (`cold_start_target` / `apply_cold_start`, +[`author_cold_start.rs:182`](../home-mixer/scorers/author_cold_start.rs#L182)): sorts all +candidate scores descending and samples `target` from the window +`ColdStartSlotMin..ColdStartSlotMax` (default `15..16`, i.e. rank 15 specifically). The chosen +candidate's score is force-raised to `max(current_score, target)` — so it lands in a real, +specific slot, not an arbitrary top position, giving it a shot at genuine impressions without +letting it dominate the feed. + +## Stage 3: author diversity decay + +Runs in `apply_author_diversity` +([`ranking_scorer.rs:697`](../home-mixer/scorers/ranking_scorer.rs#L697)), gated by +`EnableAuthorDiversity` (default `true`). Without it, an author with several high-scoring posts +in one candidate pool could occupy a disproportionate share of the feed. + +**Ranking and counting** (`compute_slate_contexts`, +[`ranking_scorer.rs:647`](../home-mixer/scorers/ranking_scorer.rs#L647)): every candidate in the +pool — all authors together — is sorted once by its current score, descending. Walking that +sorted list, each candidate is assigned `k`: how many posts from the *same author* already +appeared earlier in the sorted order. An author's single strongest post always gets `k = 0`. + +**The decay formula** (`diversity_multiplier`, +[`ranking_scorer.rs:643`](../home-mixer/scorers/ranking_scorer.rs#L643)): + +```rust +fn diversity_multiplier(decay_factor: f64, floor: f64, exponent: f64) -> f64 { + (1.0 - floor) * decay_factor.powf(exponent) + floor +} +``` + +With the defaults `AuthorDiversityDecay = 0.5` and `AuthorDiversityFloor = 0.25` +([`param.rs:229`](../home-mixer/params/param.rs#L229), +[`param.rs:235`](../home-mixer/params/param.rs#L235)), the formula is `0.75 × 0.5^k + 0.25`: + +| k (rank among this author's posts) | multiplier | +|---|---| +| 0 (author's best post) | 1.000 — no penalty | +| 1 | 0.625 | +| 2 | 0.4375 | +| 3 | 0.34375 | +| → ∞ | → 0.25 (floor, never lower) | + +**Applying it**: `pre_diversity_score × multiplier`. A prolific author's weakest post is never +docked more than 75% — the curve front-loads the penalty on the second and third posts, then +flattens, rather than punishing volume indefinitely. + +## Stage 4: out-of-network discount + +`effective_oon_weight` / `oon_applies` +([`ranking_scorer.rs:710`](../home-mixer/scorers/ranking_scorer.rs#L710) and +[`ranking_scorer.rs:776`](../home-mixer/scorers/ranking_scorer.rs#L776)) pick a flat multiplier +based on the viewer's relationship to the post: + +| condition | weight | param | +|---|---|---| +| topic feed (`topic_ids` non-empty) | `0.5` | `TopicOonWeightFactor` | +| new account, < `NewUserAgeThresholdSecs` old, ≥ 5 follows | `0.00001` | `NEW_USER_OON_WEIGHT_FACTOR` (constant) | +| everyone else, post from an unfollowed author | `0.75` | `OonWeightFactor` | +| post from a followed author | `1.0` (no discount) | — | + +One exception: if `EnableOonRescoreForInNetworkRepliesRetweets` is on (default `true`), a +reply/retweet from someone you *do* follow still gets the discount — a friend retweeting a +stranger is, functionally, stranger content reaching you. + +`NewUserAgeThresholdSecs` defaults to `0`, so the near-zero new-account tier can't actually fire +under default configuration — it's a wired-up but currently dormant lever. + +## Putting it together + +A candidate's final score is the product of all four stages applied in sequence: + +``` +score = offset_score(pos - neg) # Stage 1 + → cold-start boost (max, not ×) # Stage 2 (only for the one chosen candidate, if any) + × diversity_multiplier # Stage 3 + × oon_weight # Stage 4 +``` + +Example: two posts both reach a Stage 1 score of `0.045`. One is an out-of-network author's +second-best post in the pool (`k = 1`); one is an in-network author's best post (`k = 0`): + +``` +in-network, k=0: 0.045 × 1.000 × 1.00 = 0.0450 +out-of-network, k=1: 0.045 × 0.625 × 0.75 = 0.0211 +``` + +The out-of-network post ends up scoring less than half as much, from two independent, stackable +discounts — not one large penalty. + +## A caching note: `slate_context` can look stale on purpose + +On a paginated request (`query.has_cached_posts == true`), the entire candidate pool is the same +batch of already-scored posts served from Redis by +[`CachedPostsSource`](../home-mixer/sources/cached_posts_source.rs) — not a fresh retrieval. It +would be reasonable to assume Stage 3's `k`/rank values get reused from that cache too. They +don't: `RankingScorer` always recomputes `compute_slate_contexts` fresh against the current +candidates and scores whenever `has_cached_posts` is true +([`ranking_scorer.rs:794`](../home-mixer/scorers/ranking_scorer.rs#L794)), so the diversity +multiplier — and therefore the score — is always correct for the current request. + +What *does* stay stale on purpose: the `slate_context` field written back onto the returned +candidate. On a cached request it's populated from `stored_slate_contexts` +([`ranking_scorer.rs:680`](../home-mixer/scorers/ranking_scorer.rs#L680)) — whatever was cached +from the original, non-cached scoring pass — not from the freshly recomputed context used to +score it. This is intentional and covered by a dedicated test, +`recomputes_contexts_for_scoring_on_cached_posts` +([`ranking_scorer.rs:997`](../home-mixer/scorers/ranking_scorer.rs#L997)). `slate_context` feeds +into the request sent to `vm-ranker` +([`vm_ranker.rs:193`](../home-mixer/scorers/vm_ranker.rs#L193)), and the likely reason for keeping +it stable is to give that external reranker a consistent per-post identity across repeated page +loads, rather than a value that jitters every time the same cached post resurfaces. If you're +reading `slate_context` off a candidate for debugging, keep in mind it may not reflect the +multiplier that actually produced that candidate's `score`.