perf(models): fix /model picker UI-thread freezes and forced provider-tab re-fetches - #4615
Open
laerad777 wants to merge 3 commits into
Open
perf(models): fix /model picker UI-thread freezes and forced provider-tab re-fetches#4615laerad777 wants to merge 3 commits into
laerad777 wants to merge 3 commits into
Conversation
Owner
|
CI triage for the model-selector failures on the first run of this head (95211100724):
Local baseline check: dev head — |
…r-record rebuilds The /model picker froze the UI thread for ~350ms on selection and on every catalog change, and paid ~700ms more resolving canonical records one by one. #filterCanonicalVariants rebuilt the full candidate-key set (4,276 selector formats) once per canonical record (1,243x per query), and #resolveCanonicalVariant rebuilt a provider policy plus catalog order per record on top. Arrow-key navigation itself was innocent; the stutter came from these synchronous rebuilds landing between keypresses whenever a provider refresh or selection touched the catalog. getCanonicalModels now builds the variant filter plan once per query, and a new batch getCanonicalModelSelections resolves every record with one shared candidate set, provider policy, and catalog order. The selector and --list-models consume the batch form. On the bundled 4,276-model catalog with credentials: canonical listing 348ms->1.5ms, batch resolution ~700ms->7.5ms, post-selection refresh 351ms->40ms, per-catalog-change refresh 345ms->40ms. Lore-id: 738cccd3 Constraint: per-record resolveCanonicalModel semantics must remain identical; pinned by a whole-catalog equivalence test Tested: bun test model-registry.test.ts (270) + six model-selector suites (98) + list-models/resolver/equivalence suites (74) Not-tested: live TUI frame pacing with a real terminal Confidence: high Scope-risk: narrow Reversibility: trivial
Every provider-tab switch in /model forced a network round-trip for configured discovery providers. Two stacked causes: the selector refreshed with strategy "online" (fetch always), and configured discovery published cache rows whose discovered model ids carried no provenance fingerprint — model-manager treats id-bearing rows without matching provenance as permanently suspect and re-fetches regardless of strategy, even when the row is fresh and authoritative. The selector now uses the app-wide default strategy (online-if-uncached), and the configured-discovery path publishes the same credential-evidence+endpoint fingerprint the built-in descriptor path already records. Fresh-cache tab visits now do zero fetches; stale (24h TTL), missing, or credential/endpoint-changed caches still re-fetch. Fetch-count probe: cold visit fetches, revisit zero fetches, forced online refetches. Lore-id: 11601805 Constraint: freshness gates must survive — TTL expiry, credential evidence change, and endpoint change must each force a re-fetch Tested: model-registry (272) + discovery-manager + selector suites (298 pass); fetch-count probe cold/revisit/online Not-tested: live OCX-over-Tailscale round-trip timing Confidence: high Scope-risk: narrow Reversibility: trivial
Two CI-only failures from the tab-cache commit's blast radius:
- The Ollama Cloud tab-refresh spy pinned the old single-argument
refreshProvider call; the cache-aware path now passes the strategy
explicitly. Assertion updated to ("ollama-cloud", "online-if-uncached").
- The combined option-shape equivalence test was environmentally
vacuous locally (availableOnly scenarios saw 0 records without
credentials) and timed out in CI at 6s (ambient credentials enable
bundled providers; default test timeout is 5s), and its cross-registry
reference-identity assertion breaks on policy-applied models, which
are distinct per-registry instances. Split into four scenario tests,
seed credentials for the availability scenarios so they are
deterministic everywhere, and assert the winning provider/id instead
of object identity.
Lore-id: 6aacf30a
Constraint: scenarios must be hermetic under ambient CI credentials (GH_TOKEN et al)
Tested: model-registry + role-badge suites locally: 305 pass, 0 fail; typecheck clean
Confidence: high
Scope-risk: narrow
Reversibility: trivial
laerad777
force-pushed
the
fix/model-picker-canonical-query-perf
branch
from
August 17, 2026 00:31
9874546 to
b697743
Compare
Contributor
Author
|
Force-pushed: previous heads accidentally carried an unrelated local-only commit (
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Two perf fixes for the
/modelpicker, both reproduced on the bundled catalog (4,276 models / 1,243 canonical records):5182d07e5): selection and every catalog change blocked the main thread ~350–400ms; open-time canonical resolution cost ~700ms more.59110d0ed): every provider-tab visit forced a network round-trip, even with a fresh discovery cache.Root causes
1. Per-record O(catalog) rebuilds.
#filterCanonicalVariantsrebuilt a 4,276-entry candidate-key Set once per canonical record (1,243× per query — even when zero records survive filtering),#resolveCanonicalVariantrebuilt a provider policy + catalog order per record, and the selector re-resolved every record through the same path again. Landed with the 2026-07-22 presets refactor; only manifests for credentialed users (empty catalogs never reach the hot path), which is why CI fixtures never saw it.2. Provenance-less discovery cache rows. Configured discovery published rows whose discovered model ids carried no provenance fingerprint; the model-manager treats id-bearing rows without a matching fingerprint as permanently suspect and re-fetches under every network strategy. Combined with the selector being the app's only
online-strategy holdout, each tab visit re-fetched unconditionally.Fixes
getCanonicalModelsbuilds the variant filter plan once per query; new batchgetCanonicalModelSelectionsresolves every record with one shared candidate set, provider policy, and catalog order. Selector#materializeModelsand--list-modelsconsume the batch form. Per-recordresolveCanonicalModelsemantics unchanged.online-if-uncached); configured discovery publishes the same credential-evidence+endpoint fingerprint the built-in descriptor path already records; cache-served revisits now report the cache row's realfetchedAtinstead ofDate.now().Numbers (M5 Pro, bundled catalog)
getCanonicalModels(candidates)Freshness gates survive and are test-pinned: 24h TTL expiry, credential change, and endpoint change each force a re-fetch; a zero-fetch revisit test pins the new positive behavior.
Verification
Known tradeoffs (documented in CHANGELOG)
/model <selector>miss still forces a live probe/modelhas no explicit force-refresh keybinding (follow-up)GJC verdict
devbun checkpasses (coding-agent package: types + biome clean; full-suite single-process OOMs locally, CI shards green)b6977432a), not an earlier commit