Skip to content

perf(models): fix /model picker UI-thread freezes and forced provider-tab re-fetches - #4615

Open
laerad777 wants to merge 3 commits into
Yeachan-Heo:devfrom
laerad777:fix/model-picker-canonical-query-perf
Open

perf(models): fix /model picker UI-thread freezes and forced provider-tab re-fetches#4615
laerad777 wants to merge 3 commits into
Yeachan-Heo:devfrom
laerad777:fix/model-picker-canonical-query-perf

Conversation

@laerad777

@laerad777 laerad777 commented Aug 16, 2026

Copy link
Copy Markdown
Contributor

What

Two perf fixes for the /model picker, both reproduced on the bundled catalog (4,276 models / 1,243 canonical records):

  1. UI-thread freezes (5182d07e5): selection and every catalog change blocked the main thread ~350–400ms; open-time canonical resolution cost ~700ms more.
  2. Forced provider-tab re-fetches (59110d0ed): every provider-tab visit forced a network round-trip, even with a fresh discovery cache.

Root causes

1. Per-record O(catalog) rebuilds. #filterCanonicalVariants rebuilt a 4,276-entry candidate-key Set once per canonical record (1,243× per query — even when zero records survive filtering), #resolveCanonicalVariant rebuilt a provider policy + catalog order per record, and the selector re-resolved every record through the same path again. Landed with the 2026-07-22 presets refactor; only manifests for credentialed users (empty catalogs never reach the hot path), which is why CI fixtures never saw it.

2. Provenance-less discovery cache rows. Configured discovery published rows whose discovered model ids carried no provenance fingerprint; the model-manager treats id-bearing rows without a matching fingerprint as permanently suspect and re-fetches under every network strategy. Combined with the selector being the app's only online-strategy holdout, each tab visit re-fetched unconditionally.

Fixes

  • getCanonicalModels builds the variant filter plan once per query; new batch getCanonicalModelSelections resolves every record with one shared candidate set, provider policy, and catalog order. Selector #materializeModels and --list-models consume the batch form. Per-record resolveCanonicalModel semantics unchanged.
  • Selector tab refresh uses the app-wide default strategy (online-if-uncached); configured discovery publishes the same credential-evidence+endpoint fingerprint the built-in descriptor path already records; cache-served revisits now report the cache row's real fetchedAt instead of Date.now().

Numbers (M5 Pro, bundled catalog)

Path Before After
getCanonicalModels(candidates) 348ms 1.5ms
Batch canonical resolution ~700ms 7.5ms
Post-selection refresh 397ms 13.6ms
Per-catalog-change refresh 407ms 12ms
Provider-tab revisit (fresh cache) network round-trip 0 fetches (incl. cross-process)

Freshness gates survive and are test-pinned: 24h TTL expiry, credential change, and endpoint change each force a re-fetch; a zero-fetch revisit test pins the new positive behavior.

Verification

  • 276 tests pass (model-registry incl. 3 new equivalence/pin tests, discovery-manager); typecheck clean; PTY dogfood on both builds with a real OCX config showed functional parity, zero errors
  • Whole-catalog equivalence tests: batch ≡ per-record loop across availableOnly × candidate-subset × whitespace-sessionId scenarios
  • Independent review: architect + critic passes on both commits (ship / approve; all findings addressed — sessionId trim parity, matrix breadth, fetchedAt honesty, changelog accuracy)

Known tradeoffs (documented in CHANGELOG)

  • Provider tabs reflect server-side catalog changes at most once per 24h unless credentials/endpoint change; /model <selector> miss still forces a live probe
  • /model has no explicit force-refresh keybinding (follow-up)
  • OAuth token rotation invalidates the discovery cache (pre-existing identity design; safe over-fetch direction)

GJC verdict

gajae.pr-review-verdict.v1 needs-human sha256:b64a204bf6c0a53d8822bd92fc20ab63e2bf3c8af488639f785396c015e2d0d7 reviewer:human reviewer-id:laerad777 evidence:local suites 305 pass on b6977432a (model-registry + role-badge); prior head CI test shards green

  • Target branch is dev
  • bun check passes (coding-agent package: types + biome clean; full-suite single-process OOMs locally, CI shards green)
  • Tested locally
  • CHANGELOG updated
  • Verdict above matches the exact PR head (b6977432a), not an earlier commit

@Yeachan-Heo

Copy link
Copy Markdown
Owner

CI triage for the model-selector failures on the first run of this head (95211100724):

  • model-selector-role-badge-thinking.test.ts → 1 fail: "refreshes Ollama Cloud using provider id instead of tab label" — expected knownProviderIds without "online-if-uncached", received one extra entry (line 518). This looks directly related to this PR's provider-tab re-fetch changes (model-selector.ts is in the diff), not a flake: the refresh path now surfaces the online-if-uncached pseudo-provider id into the canonical list.
  • The other 11 failed jobs are downstream aggregates (evidence producer / ts-build fan-out) of that single test-file failure; the sibling run on the same head already passed everything else.

Local baseline check: dev head c83ffe3 passes 30/30 on that file, so the delta is introduced by this branch. Suggest either filtering online-if-uncached out of knownProviderIds in the refresh path or updating the pinned expectation if the new id is intentional.


[repo owner's gaebal-gajae (clawdbot) 🦞]

…r-record rebuilds

The /model picker froze the UI thread for ~350ms on selection and on
every catalog change, and paid ~700ms more resolving canonical records
one by one. #filterCanonicalVariants rebuilt the full candidate-key set
(4,276 selector formats) once per canonical record (1,243x per query),
and #resolveCanonicalVariant rebuilt a provider policy plus catalog
order per record on top. Arrow-key navigation itself was innocent; the
stutter came from these synchronous rebuilds landing between keypresses
whenever a provider refresh or selection touched the catalog.

getCanonicalModels now builds the variant filter plan once per query,
and a new batch getCanonicalModelSelections resolves every record with
one shared candidate set, provider policy, and catalog order. The
selector and --list-models consume the batch form. On the bundled
4,276-model catalog with credentials: canonical listing 348ms->1.5ms,
batch resolution ~700ms->7.5ms, post-selection refresh 351ms->40ms,
per-catalog-change refresh 345ms->40ms.

Lore-id: 738cccd3
Constraint: per-record resolveCanonicalModel semantics must remain identical; pinned by a whole-catalog equivalence test
Tested: bun test model-registry.test.ts (270) + six model-selector suites (98) + list-models/resolver/equivalence suites (74)
Not-tested: live TUI frame pacing with a real terminal
Confidence: high
Scope-risk: narrow
Reversibility: trivial
Every provider-tab switch in /model forced a network round-trip for
configured discovery providers. Two stacked causes: the selector
refreshed with strategy "online" (fetch always), and configured
discovery published cache rows whose discovered model ids carried no
provenance fingerprint — model-manager treats id-bearing rows without
matching provenance as permanently suspect and re-fetches regardless
of strategy, even when the row is fresh and authoritative.

The selector now uses the app-wide default strategy
(online-if-uncached), and the configured-discovery path publishes the
same credential-evidence+endpoint fingerprint the built-in descriptor
path already records. Fresh-cache tab visits now do zero fetches;
stale (24h TTL), missing, or credential/endpoint-changed caches still
re-fetch. Fetch-count probe: cold visit fetches, revisit zero fetches,
forced online refetches.

Lore-id: 11601805
Constraint: freshness gates must survive — TTL expiry, credential evidence change, and endpoint change must each force a re-fetch
Tested: model-registry (272) + discovery-manager + selector suites (298 pass); fetch-count probe cold/revisit/online
Not-tested: live OCX-over-Tailscale round-trip timing
Confidence: high
Scope-risk: narrow
Reversibility: trivial
Two CI-only failures from the tab-cache commit's blast radius:

- The Ollama Cloud tab-refresh spy pinned the old single-argument
  refreshProvider call; the cache-aware path now passes the strategy
  explicitly. Assertion updated to ("ollama-cloud", "online-if-uncached").

- The combined option-shape equivalence test was environmentally
  vacuous locally (availableOnly scenarios saw 0 records without
  credentials) and timed out in CI at 6s (ambient credentials enable
  bundled providers; default test timeout is 5s), and its cross-registry
  reference-identity assertion breaks on policy-applied models, which
  are distinct per-registry instances. Split into four scenario tests,
  seed credentials for the availability scenarios so they are
  deterministic everywhere, and assert the winning provider/id instead
  of object identity.

Lore-id: 6aacf30a
Constraint: scenarios must be hermetic under ambient CI credentials (GH_TOKEN et al)
Tested: model-registry + role-badge suites locally: 305 pass, 0 fail; typecheck clean
Confidence: high
Scope-risk: narrow
Reversibility: trivial
@laerad777
laerad777 force-pushed the fix/model-picker-canonical-query-perf branch from 9874546 to b697743 Compare August 17, 2026 00:31
@laerad777

Copy link
Copy Markdown
Contributor Author

Force-pushed: previous heads accidentally carried an unrelated local-only commit (43bf5e9ce, packages/agent work) because the branch was rebased onto an unpushed local dev tip. New head b6977432a contains exactly the three perf/test commits against origin/dev (c83ffe3d7); PR diff is now coding-agent-only (18 files).

  • CI test failures from the first run are fixed (spy arity + hermeticity/timeout of the equivalence scenarios — seeded credentials, split tests, winner-semantic assertions).
  • Verdict: needs-human per template (no authenticated approving review for this head; author has no write access). Happy to address any review feedback.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants