feat(search): discount the exact-name bonus by corpus frequency (#982)

[maxmilian]

Implements the corpus-frequency discount from #982 — the general lever #746 floated and left unbuilt.

Read this first — which half of #982 this is. #982 asks for two deliberately-distinct levers, and this PR is only one of them. #1463 is the one that fixes #982's 8-file minimal reproduction; this one does not, and cannot, because the repro's usage is rare (2 symbols of ~25) and this lever keys on a name being common. The two cover different shapes on purpose — details and measurements in "Scope" at the bottom. If you are evaluating against the repro, start with #1463.

The change

nameMatchBonus handed a flat 80 (whole query === name) / 60 (a token of a multi-word query === name) no matter how many symbols carry that name. STOP_WORDS deliberately doesn't filter common symbol names, so a bare usage / get / status collected the full bonus and outranked product code that answers the query but doesn't literally contain the token.

nameMatchIdfScale(df, total) = log(1 + total/df) / log(1 + total)

• a unique name yields exactly 1 — the common case is bit-for-bit unchanged
• weight decays as the name spreads across the corpus
• floored at 0.60 — measured, not picked; see below
• only the two exact-name arms are scaled. Prefix/substring bonuses are already small and length-scaled and never produced the crowd-out
• corpus stats reach the scorer through an optional parameter, so nameMatchBonus stays pure and every existing caller behaves exactly as before when stats aren't supplied

Because findRelevantContext retrieves through searchNodes, the explore path inherits this.

The two things I asked you to decide, now answered with measurements

I left this draft open on two questions rather than guess. Both are settled, and both answers turned out different from what I expected — the second one contradicts the concern I raised.

1. The floor: 0.25 was inert. It is now 0.60, derived from a sweep.

Swept the floor 0→1 over the top-25 corpus-common names of five indexed repos — gin (2,531 nodes), Alamofire (4,512), codegraph itself (9,230), excalidraw (11,161), django (62,080) — scoring two opposing constraints:

• B — must-still-rank: a query that is a common name must still return that name first. This is the constraint the floor exists for.
• A — crowd-out: share of a mixed query's top-10 taken by symbols literally named the one generic token in it. This is the explore/query relevance: a generic token's exact name-match overboosts in peripheral dirs — follow-up to #746 #982 pathology.

The first finding is that 0.25 never did anything. Real corpora don't drive the raw scale that low — the minimum reached is 0.36–0.47:

repo	nodes	commonest name df	lowest raw scale
gin	2,531	63	0.474
Alamofire	4,512	173	0.392
excalidraw	11,161	387	0.364
django	62,080	1,097	0.367

Floors of 0, 0.10, 0.15, 0.20, 0.25 and 0.30 produce byte-identical sweep results on all five repos. The number I flagged as "the one I picked rather than derived" was dead code.

The second finding is that the real failures sit higher, and they are indefensible ones. With the floor inert, searching Alamofire for request (df 173 → scale 0.392) demoted the symbol actually named request to rank 3, behind requests — a prefix match. Searching for alamofire (scale 0.551) put it at rank 4, behind AlamofireExtended. Same shape on codegraph itself: vitest (scale 0.445) fell to rank 2. The user typed the name; burying it under a longer name that merely starts with it is never right.

That gives a principled bound, not a taste call: the prefix arm pays 10 + 30 * ratio with ratio < 1, so it approaches but never reaches 40. A floored whole-query exact match pays 80 * floor. 80 * 0.6 = 48 > 40 — above the prefix arm at any corpus frequency. The sweep agrees empirically: 0.60 is the lowest tested value restoring exact-name recall@1 to the undiscounted baseline on all five repos.

Exact-name recall@1 (top-25 common names; floor 1.0 = discount off):

floor	gin	Alamofire	codegraph	excalidraw	django
0.25	0.88	0.92	0.96	1.00	1.00
0.50	0.88	0.96	1.00	1.00	1.00
0.55	0.88	0.96	1.00	1.00	1.00
0.60	0.88	1.00	1.00	1.00	1.00
1.00 (off)	0.88	1.00	1.00	1.00	1.00

gin's 0.88 is identical at every floor including discount-off, so it is not this change — the three misses are framework route nodes (get /, get /example) that the query tokenizer splits oddly. Pre-existing, untouched, and I'd rather flag it than let it read as fallout here.

And the cost of moving 0.25 → 0.60 is small. Crowd-out in top-10 (lower = more relief):

repo	floor 0.25	floor 0.60	1.00 (off)	relief kept
gin	0.600	0.600	0.612	100%
Alamofire	0.708	0.712	0.780	94%
codegraph	0.604	0.612	0.820	96%
excalidraw	0.684	0.692	0.836	95%
django	0.700	0.716	0.888	91%

~95% of the relief, and the exact-name regression is gone. Sweep harness and per-name failure dumps are reproducible from the numbers above; say the word and I'll land them under scripts/agent-eval/.

Honest caveat on metric A: "share of top-10 literally named the generic token" is a proxy. It has no ground-truth notion of the right answer — it measures the pathology's signature, not relevance. It is directionally sound for this specific failure and I would not read the absolute values as a quality score.

2. The hot-path cost: the COUNT(*) I worried about is free. The per-name lookup was the real problem — now fixed.

I flagged one COUNT(*) per searchNodes and offered to cache it on the instance. Measured, that was the wrong thing to worry about: it costs 0.002 ms. No cache needed, and I'd rather not add invalidation machinery for it.

The cost I did not flag is where the time went. nameCorpusStats counted names with:

SELECT COUNT(*) FROM nodes WHERE name = ? COLLATE NOCASE

which matches neither index — not idx_nodes_name (BINARY collation) nor idx_nodes_lower_name (expression form). EXPLAIN QUERY PLAN confirms SCAN nodes USING COVERING INDEX idx_nodes_name: a full scan per distinct candidate name, so ~50 scans on every search, growing with the corpus.

Written as lower(name) = ? it seeks idx_nodes_lower_name (SEARCH … (<expr>=?)). Per search, 50 distinct candidate names:

repo	nodes	COLLATE NOCASE	lower(name)	speedup
gin	2,531	3.17 ms	0.08 ms	42×
Alamofire	4,512	5.63 ms	0.08 ms	75×
excalidraw	11,161	14.09 ms	0.08 ms	182×
django	62,080	79.33 ms	0.09 ms	912×

The seek is flat in corpus size; the scan is linear in it. getNodesByLowerName in the same file already used lower(name) = ?, so the COLLATE NOCASE form I reached for was also out of step with the established idiom here. With this fixed the whole feature costs ~0.08 ms per search on a 62k-node repo, so the memoized-per-search design needs no further caching.

Adjacent, not touched by this PR: the pre-existing exact-match supplement in searchNodesFTS uses the same non-indexable name = ? COLLATE NOCASE. It's LIMIT 20 so it short-circuits when matches exist, but a query term with no exact match scans the table. Happy to fix it in a separate PR — it isn't mine to change here.

Agent A/B

The honest summary: the agent-level A/B cannot resolve this change. Deterministic probes can, and do.

I ran both harnesses CLAUDE.md asks for, --model sonnet --effort high, 2 runs/arm.

First attempt was void and I'm reporting it rather than dropping it. I ran ab-new-vs-baseline.sh on an implementation task ("add a system check for Meta ordering…"). Both arms answered it with 16–44 Bash greps and zero codegraph calls — the agent never queried the index, so the arms differed by run-to-run noise alone (176s/$0.87 vs 418s/$2.70). That gap looks like a large win for this branch and is nothing of the sort. Discarded.

Isolating A/B — new build vs 572d22b, both codegraph-on, flow questions (2 runs/arm):

repo	arm	codegraph calls	Read	duration
django	new	2, 2	0, 0	27s, 35s
django	baseline	2, 2	0, 0	23s, 35s
excalidraw	new	3, 1	0, 0	38s, 34s
excalidraw	baseline	1, 1	0, 0	33s, 23s

Parity. Zero file reads in all 8 arms, and the between-arm spread is smaller than the within-arm spread (the excalidraw new arm alone ran 1 and 3 explores on identical input). I can't claim an agent-level win from this and I'm not going to.

Pass bar — with vs without codegraph (2 runs/arm), unaffected by this change but confirming no regression:

repo	arm	tool calls	Read	duration
django	with	2, 2	0, 0	29s, 36s
django	without	7, 7	4, 4	35s, 39s
excalidraw	with	3, 1	0, 0	49s, 30s
excalidraw	without	21, 22	10, 11	21s, 22s

django clears it on every axis. excalidraw's without-arm is faster while doing 10–11 reads — the small-repo floor effect the README already documents, not something this branch caused.

Why the A/B is the wrong instrument here, and what is the right one. A flow question is answered from buildFlowFromNamedSymbols over a precise symbol bag; the exact-name tie-break this PR touches barely participates. So I pointed a deterministic probe at the condition the lever actually keys on — a query containing a corpus-common symbol name — and diffed the top-10 with the discount on vs off:

django (62,080 nodes) — real reordering, in the intended direction:

query	discount off	discount on
setup handling during request processing	setup, request, setup, then 7× test setUp	request ×5, Request, request@tests ×3 — every test setUp leaves the top-10
where is setup configured at startup	setup ×3, configured, 6× setUp	configured promoted to rank 1, setUp demoted
where is meta configured at startup	Meta ×10 — the whole page	configured promoted to rank 1

excalidraw (11,161 nodes) — inert. Every query's ordering is byte-identical; only absolute scores shift uniformly (75→51). Its corpus-common names are import module nodes (react, clsx, vitest) that all share one name, so a uniform discount cannot reorder them among themselves.

That is a coherent result rather than a flattering one: the lever fires on a large, symbol-name-rich corpus and is a no-op elsewhere. It never reordered anything for the worse in these probes. But it is a retrieval tie-break whose effect is below the resolution of the agent A/B this repo gates on, so I'd rather hand you that finding plainly than dress up a null result. Left as a draft on that basis — your call whether the deterministic evidence is the right bar for a change of this shape.

Tests

__tests__/name-match-idf.test.ts, 14 tests — all fail on main, pass here:

• nameMatchIdfScale: unique name → exactly 1; monotone decay (strict until the floor binds); a very common name discounted to the floor but non-zero; degenerate inputs (df=0, total=1, NaN, df > total) safe
• the floor invariant: 80 * FLOOR exceeds the prefix arm's supremum of 40, plus the concrete Alamofire request vs requests inversion that motivated it
• the floor is above what real corpora reach: pins nameMatchIdfScale(1097, 62080) and (173, 4512) at the floor, so a future "simplification" back to 0.25 fails loudly instead of silently going inert
• nameMatchBonus: unchanged without corpus stats; unchanged for a rare name; discounted for a common one; prefix/substring arms untouched
• the end-to-end ranking flip on explore/query relevance: a generic token's exact name-match overboosts in peripheral dirs — follow-up to #746 #982's layout, plus a control that a query for usage still surfaces usage() in the top 3
• the explicit rare-name test documenting the scope limit below

Ranking-adjacent suites, all green at the new floor: context-ranking, explore-corroboration-ranking, explore-nl-stopword-collision, explore-result-count, symbol-lookup, same-name-disambiguation, field-name-retrieval, search-query-parser — 61 passed. tsc --noEmit clean.

Scope — what this does NOT fix

#982's 8-file minimal reproduction is not fixed by this change. In that repro only two symbols are named usage out of ~25 nodes. The token is rare there, so the IDF scale is ≈0.8 — nearly inert, which is correct behaviour for a corpus-frequency lever. Measured: the helpers still land at 62.8 vs the top product symbol at 51.2.

The repro demonstrates the mechanism (the exact-name bonus dominates) but not the condition this lever keys on (the name being common). What fixes that shape is the issue's complementary, deliberately-separate path lever — user-extensible de-prioritization via codegraph.json — which is #1463. #982 explicitly asks for the two to stay distinct, and there's a test here pinning the rare-name case so nobody later "fixes" the inertness by removing the floor.

The two PRs are independent, both off main, and touch the same region of scorePathRelevance's neighbours — whichever lands second needs a light rebase, in either order.

[maxmilian]

Pushed 31bdefc and rewrote the description. Both open questions are answered with measurements now — and I want to lead with the part that argues against this PR rather than bury it.

The agent A/B does not support this change. It shows parity.

ab-new-vs-baseline.sh against 572d22b, both arms codegraph-on, --model sonnet --effort high, 2 runs/arm on django and excalidraw: 2 vs 2 and 3/1 vs 1/1 codegraph calls, 0 file reads in all 8 arms, durations overlapping. The between-arm spread is smaller than the within-arm spread — the excalidraw new arm alone ran 1 and 3 explores on identical input. There is no agent-level win here to claim.

A first attempt was worse than useless and I'd rather report it than drop it: run on an implementation task, both arms answered with 16–44 Bash greps and zero codegraph calls, producing a 176s/$0.87 vs 418s/$2.70 split that looks like a huge win for this branch and is pure noise. Discarded.

What does resolve the change is a deterministic probe pointed at the condition the lever keys on — a query containing a corpus-common symbol name. On django (62k nodes) the reordering is real and in the intended direction: where is meta configured at startup returns ten Meta nodes with the discount off and promotes configured to rank 1 with it on; setup handling during request processing drops every test setUp out of the top-10. On excalidraw it is completely inert — identical ordering, only a uniform score shift — because its common names are import module nodes that all share one name.

So: fires on a large symbol-name-rich corpus, no-op elsewhere, nothing reordered for the worse. But it is a retrieval tie-break whose effect sits below the resolution of the A/B this repo gates on, so I'm leaving it as a draft. Your call whether deterministic evidence is the right bar for a change of this shape — I didn't want to flip it to ready on evidence that doesn't meet the bar you actually set.

Two things worth your attention regardless:

• The floor was dead code. Real corpora never drive the raw scale below ~0.36 (django's commonest name spans 1097 of 62080 nodes), so 0.25 never once bound — floors of 0 through 0.30 give byte-identical results on all five repos I swept. The failures that do exist sit higher and are indefensible: searching Alamofire for request demoted the symbol named request below requests, a prefix match. Now 0.60, which puts 80 * floor = 48 above the prefix arm's supremum of 40 so that inversion is impossible at any frequency, and restores exact-name recall@1 to the undiscounted baseline on all five repos while keeping ~95% of the crowd-out relief.
• A real hot-path regression I introduced, now fixed. The COUNT(*) I flagged as the thing to watch costs 0.002 ms — I was worried about the wrong query. name = ? COLLATE NOCASE matches neither idx_nodes_name (BINARY) nor idx_nodes_lower_name (expression), so it full-scanned per candidate name: 79 ms per search on django, growing with the corpus. As lower(name) = ? it seeks the index and is flat at ~0.08 ms everywhere. getNodesByLowerName in the same file already used that form, so my version was also out of step with the local idiom.

Unrelated and not touched here: the pre-existing exact-match supplement in searchNodesFTS has the same non-indexable COLLATE NOCASE. LIMIT 20 short-circuits it when matches exist, but a term with no exact match scans the table. Happy to send that separately if you want it.

Tests are 14 now, including one pinning nameMatchIdfScale(1097, 62080) at the floor so a future "simplification" back to 0.25 fails loudly instead of silently going inert. Ranking-adjacent suites 61 passed, tsc --noEmit clean.

[colbymchenry]

With #1463 and #1542 both on main now, here's what this needs to come out of draft — one real blocker, one small alignment, and two things you were tracking that are now moot.

The blocker: the two discounts compose, and the composition breaks your own invariant. This PR's floor derivation and #1463's damping derivation each pin "an exact name the user typed never loses to a mere prefix match," and each clears it in isolation: 80 × 0.60 = 48 > 40 here, 80 × 0.75 − 15 = 45 > 40 there. But once this rebases onto main, the natural merge stacks the multipliers on a name that is both corpus-common and inside a deprioritized tree: 80 × 0.60 × 0.75 − 15 = 21, well under the prefix arm's supremum of 40. The general condition is combined scale > 55/80 = 0.6875 (a deprioritized node also carries the −15 path penalty). Neither suite catches it because each pins its invariant with the other lever off. Two clean fixes — your pick: floor the combined multiplier above 0.6875, or apply the stronger of the two discounts rather than their product (min, not ×). Either way, add one composed test: a deprioritized, corpus-common exact name still beats a prefix match.

The alignment: nameCorpusStats' per-name count lowers the parameter in JavaScript and compares it against SQLite's lower(name) — the exact ASCII-vs-Unicode asymmetry your own #1542 write-up documents. The failure direction is benign (a non-ASCII common name just goes undiscounted), but with #1542 merged, spell it lower(name) = lower(?) with the raw name so the file stays on one idiom.

Moot: the "adjacent, not touched" searchNodesFTS scan you footnoted is fixed on main (that was #1542), and the rebase will pick up #1463's plumbing in the same searchNodes region you're touching — expect a small textual conflict there.

On the merge decision itself: the direction is right, and the floor sweep was the strongest part of the draft — real corpora never driving the raw scale below ~0.36 (making the old 0.25 dead code), and the exact-name-loses-to-prefix failures at low floors, are findings that only come from running the thing. Since this changes default ranking (unlike #1463), the bar is: rebase, fix the composed bound, and re-run the deterministic probes on the rebased tree so the numbers describe what would merge. The parity agent A/B doesn't block it, for the same reason it didn't block #1463 — wrong instrument for effects this size. Do that and I'm happy to promote it.

[maxmilian]

Rebased and pushed (fbf36af). Three of the four things you asked for are done and green; the fourth — re-running the probes on the rebased tree — turned up something that I think changes the decision, so I've left this in draft rather than calling it ready.

The composed bound

Fixed, but not the way you suggested, because the second option doesn't clear its own bound: min(0.6, 0.75) is 0.6, and 80 × 0.6 − 15 = 33, still under 40. Taking the stronger discount isn't enough when the stronger one is already at the floor.

So it's the first option — a floor on the combined multiplier. Working the rounding through: the check is round(80 × s) − 15 > 40, so s ≥ 0.69375, and 0.70 is the next clean value above it, leaving round(56) − 15 = 41. That's a one-point margin, which is why the derivation is in the constant's doc comment rather than just the number.

Structurally this meant moving DEPRIORITIZED_NAME_BONUS_SCALE into query-utils.ts (re-exported from queries.ts for the existing importers) — only that module sees both multipliers, so only it can hold the bound they share. The non-exact arms keep #1463's blanket damping exactly as before; I checked each arm for equivalence.

The composed test is in __tests__/composed-name-bonus.test.ts. I falsified it rather than trusting it: with the floor set back to 0 it fails with expected 21 to be greater than 29 — your 21, from a real ranking comparison rather than restating the arithmetic.

The nameCorpusStats alignment is in too. Worth noting the parameter was the actual bug there, not just the SQL: it was passing the JS-lowered key, so lower(name) = ? was comparing against an already-lowered value. It's lower(name) = lower(?) with the raw name now, and since lower(?) is a constant expression it still seeks the index rather than scanning.

What the probes found

Full suite on the rebased tree: 61 failures against 57 on a clean main worktree (the shared 57 are the CLI tests needing a build — same on both). Diffing the two runs by test name, exactly 4 fail only on this branch, and all 4 are in files added to main after this branch's point:

• __tests__/explore-allocation-1500.test.ts (bd86ad2) — the CG-12 gate and the allocation snapshot
• __tests__/explore-cross-call-dedup.test.ts (ab38d1f) — both cross-call cases

I isolated the cause rather than assuming it was the new floor. Setting the floor to 0 (pure product) gives the same 4 failures, so it isn't the floor. Replacing the corpus argument with undefined gives 41/41 green, so it is the corpus discount itself.

Here's what actually reaches the agent, same fixture, same query:

	files delivering bytes
without the discount (main today)	usecase/payroll/cycle.go 5912, domain/payroll/payslip.go 4094, store/payslipstore/store.go 2291, usecase/payroll/payslip_builder.go 2209
with the discount (this branch)	usecase/payroll/cycle.go 5379, domain/payroll/payslip.go 4094, transport/httpapi/payroll_handler.go 2813, internal/gen/fkit/payroll/payroll_cycle.go 2088

The discount puts a generated CRUD file back into the envelope and pushes both hand-written files out. That's the #1500 behaviour, arriving through the change meant to reduce it. The CG-10 gates still pass — the answer/generated share split holds — so what breaks is CG-12 specifically: payslip_builder.go reaches the response only via the slot hand-off, and the discount moves it out of reach.

The mechanism, as far as I can tell: the fixture deliberately gives half the generated tree names that collide with the hand-written layer. When a name collides, both copies get discounted, so their relative order is preserved — but the hand-written file's absolute score drops, and it loses to a generated file whose name has no twin and is therefore not discounted at all. The discount penalises having your name copied, and being copied is what the hand-written original looks like from the index's point of view.

I tried the obvious narrowing — raising the nameMatchIdfScale guard from df <= 1 to df <= 2, on the theory that two symbols sharing a name isn't "corpus-common". It doesn't help (same 4 failures) and it breaks one of this PR's own tests, so those collisions are already at df >= 3.

Where that leaves it

I don't think I should pick the direction here. Making the discount immune to this would mean changing its shape, and that invalidates the floor sweep the 0.60 came out of — five repos, measured against the undiscounted baseline. That's your call, not a patch I should land quietly:

1. narrow where the discount applies (some notion of "common relative to the corpus" that a small collision cluster doesn't trip), and re-run the sweep;
2. accept the ranking shift and update the CG-12 expectation, if the discount's win elsewhere outweighs it on your fixtures;
3. or conclude this belongs behind a flag rather than in default ranking, given it's the default-ranking bar that made this visible in the first place.

Staying in draft until you say which. The rebase, the composed bound, the composed test and the alignment are all in fbf36af regardless — none of that is contingent on the answer.