feat: structural scout platform + search determinism fixes (clean port onto main)#169
Open
jahala wants to merge 3 commits into
Open
feat: structural scout platform + search determinism fixes (clean port onto main)#169jahala wants to merge 3 commits into
jahala wants to merge 3 commits into
Conversation
… integration line Deliberate port-not-merge: the integration workbench accumulated multiple experiment lines (footer variants, git-fusion ranking, a transitive trace tool — all A/B-losers or unvalidated); this branch rebuilds only the wanted work directly on main. Ported: - Local-inference layer (src/infer: CoreML-backed reranker + embedder, compiled-model cache, format-versioned embed index, on-demand pool scoring, size-independent queries, explicit --warm) behind the `infer` feature; `tilth model pull` (+optional model-pull-http). - The scout pipeline in lib.rs: locate ∪ embed-recall pool, noise filter, rrf(CE, embed) on flattened symbol texts, ce1_winner gate, grok trace skeleton; CLI `tilth scout` (+--hook, --warm), `tilth_scout` MCP tool, `install --with-scout` (absolute hook path + timeout). - A trimmed locate core (density ranking, symbol ranking, test/nav-noise classifiers) — the footer engine, git-fusion, and their A/B-losing variants are deliberately NOT ported. - Bisect kill-switches (TILTH_NO_RDALLOC/QTRUNC/OGATE/THINWRAP) with race-free env helper + old-behavior tests. - Determinism fixes: tilth_files collect-sort-cap (selection was thread-timing dependent), blast-radius caller ordering, volatile timing stripped from MCP scout results. Verified: 583+ tests green, clippy clean (default + infer), release fire-map matches the validated workbench behavior (deepchain/kong/comrak fire with correct anchors; easy prompts silent; hook emits skeleton-only or nothing), tilth_files byte-deterministic across runs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…osure The parallel search walkers (symbol, content/regex, batch callers) quit on a racy shared counter, making the discovered match SET thread-timing dependent: identical calls returned different totals and different displayed matches (observed: 6 distinct outputs over 8 identical runs), and a regex search silently surfaced matches from 1 file when 12 contained them — the walk quit before reaching the rest, and the result read as complete. - Remove the early-quit from all three walkers: the walk always completes, per-file work is already bounded (size caps, minified skips, bloom prefilter), and ranking + display caps apply deterministically after the walk. Common-token searches remain fast (~40 ms on this repo). - Keep the callers exists-check quit (boolean outcome — order-independent). - Honest rendering: the search header now says "N of M matches" whenever the display cap bit, and content/regex results (which carry no per-facet totals) emit the global "... and K more matches" tail on the faceted path. Failing-tests-first: completeness (a file beyond the old quit threshold must be counted), run-to-run determinism, and rendered disclosure — all red on the old code, green now. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The k configuration — the first to beat the native-tool baseline — carried two instruction lines with measured effect (grok adoption 57%→78% on trace tasks) that never landed in the prompts lineage: the tilth_grok call-chain steering and the multi-symbol/multi-section batching directive. Both are now in mcp-base.md, byte-locked so they cannot silently drift out again, and AGENTS.md is regenerated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
A clean port of the validated scout line onto main — three commits, rebuilt from main tip rather than merged from the integration workbench:
048e8e3): optional local-inference layer (src/infer/, Cargo featureinfer, zero default-build weight),tilth scoutCLI (--job context|rerank,--warm,--hook), reactivetilth_scoutMCP tool,tilth model pull, trimmed locate core (the footer/git-fusion experiments are deliberately left behind),TILTH_NO_*bisect kill-switches, determinism fixes (deterministictilth_files, sorted blast radii, no volatile bytes in MCP results), size-independent embed index (.gitignore-aware walker, on-demand pool embedding, warm-only union, format-versioned cache), CoreML compiled-model cache (fixes the$TMPDIRleak).d67fc1b): removes the racy early-quit in all three parallel search walkers — identical searches previously returned different match sets run-to-run, and capped regex/content results could silently read as complete. Walks now complete, results are byte-deterministic, headers disclose "N of M matches".9127759): restores the two measured steering lines (grok for call-chains; batching for multi-symbol/multi-section calls) with updated byte-lock tests.Receipts
tilth install --with-scout) pending the re-aimed value test.clippy -D warningsclean,fmt --checkclean; MCP fire-map verified on the release--features inferbuild.Not in this PR
No version bump (release PR comes separately per the release workflow). No footer (measured net-negative; not ported). No hook-by-default. Follow-up branches stack on this and arrive as separate PRs: grok qualified-name resolver, language-coverage fixes (Ruby/C/C++/Go), response-format v2.
🤖 Generated with Claude Code