Add ReusableState for allocation-free incremental FST walks - #27
Open
oc-engteam wants to merge 1 commit into
Open
Add ReusableState for allocation-free incremental FST walks#27oc-engteam wants to merge 1 commit into
oc-engteam wants to merge 1 commit into
Conversation
AcceptWithVal/IsMatchWithVal decode each source state with a nil prealloc, so they allocate a fresh fstStateV1 on every transition. Callers that walk the FST one byte at a time over many overlapping prefixes — e.g. building a CJK segmentation DAG that restarts from every input position — therefore allocate once per transition. In a real FST-backed CJK tokenizer this was ~69% of all allocations (and ~2x the wall-clock once removed). Add an opaque ReusableState plus AcceptWithValState/IsMatchWithValState that thread a reusable *fstStateV1 through decoder.stateAt. fstStateV1 only references the FST data (no owned slices), so a single ReusableState backs an entire walk with zero per-transition allocation. Results are identical to the stock methods; the existing API is untouched. Tests: TestReusableStateMatchesStock (parity across match/prefix/absent keys) and TestReusableStateNoAlloc (AllocsPerRun == 0). Signed-off-by: Engineering Team <oc-engteam@codetrek.work> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
oc-engteam
added a commit
to codetrek/haystack
that referenced
this pull request
Jun 18, 2026
…85) Vendor vellum v1.0.10 (library only) under core/third_party/vellum with a relative replace, adding AcceptWithValState/IsMatchWithValState that thread a reusable *fstStateV1 through decoder.stateAt. The fstcjk segmenter holds one ReusableState per Cut (pooled) — eliminating the per-FST-transition state allocation that a memprofile pinned at ~69% of CJK-tokenization allocs. Byte-identical to gse (fidelity tests + the FST-vs-gse CI gate pass). BenchmarkCJKTokenizeForIndex: 1756->396 allocs/op (-77%), 232K->22K B/op (-90%), 138us->66us (2.1x faster). Also excludes third_party from the repo-wide gofmt check (vendored upstream code). The 2 reuse methods are proposed upstream (blevesearch/vellum#27); if accepted, the vendored copy can be dropped for a version bump.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
AcceptWithVal/IsMatchWithValdecode each source state with anilprealloc, so they allocate a freshfstStateV1on every transition. Callers that walk the FST one byte at a time over many overlapping prefixes — e.g. building a CJK segmentation DAG that restarts the walk from every input position — pay one allocation per transition.In a real FST-backed CJK tokenizer (replacing a cedar-trie dictionary with a vellum FST), a memprofile showed
decoderV1.stateAtaccounting for ~69% of all allocations; threading a reusable state removed them and made tokenization ~2× faster wall-clock (allocs/op 1756→396, B/op 232K→22K).Change
fstStateV1only references the FST data (no owned slices), so a single reusable instance can back an entire walk. This PR adds:ReusableState— an opaque, caller-owned decode buffer (NewReusableState()).AcceptWithValState(addr, b, rs)andIsMatchWithValState(addr, rs)— identical behavior to the existing methods, but decode intorsinstead of allocating.The existing API is untouched; this is purely additive. A
ReusableStateis not safe for concurrent use (documented), matching the per-goroutine pattern ofReader.Tests
TestReusableStateMatchesStock— the*Statevariants return identical(output, final)toAcceptWithVal/IsMatchWithValacross matching, prefix, and absent keys, reusing oneReusableStateacross keys.TestReusableStateNoAlloc—testing.AllocsPerRunof a full walk is0.Disclosure: drafted with AI assistance; reviewed and tested locally (
go testgreen,gofmtclean).