Skip to content

main is broken: concepts_search_fts FTS5 table is used but never created (semantic merge conflict between #194 and #233) #274

Description

@mauripunzueta

Summary

main currently fails Test Rust. crates/hts/tests/value_set_ops.rs::expand_is_a_filter_with_text_filter_still_nests errors with:

no such table: concepts_search_fts

The concepts_search_fts FTS5 table is read from and written to in crates/hts/src/backends/sqlite/value_set.rs, but its CREATE VIRTUAL TABLE does not exist anywhere in the repository:

$ grep -rn "concepts_search_fts" --include=*.rs .
crates/hts/src/backends/sqlite/value_set.rs   (10 usages: SELECT / MATCH / bm25 / DELETE / INSERT)

$ grep -rn "CREATE VIRTUAL TABLE.*concepts_search_fts" .
(no matches)

The sibling tables (concepts_fts, concepts_word_fts, implicit_expansion_fts, …) are all declared in crates/hts/src/backends/sqlite/schema.rs. This one was never added.

populate_concepts_search_fts_for_system() propagates the error with ?, so the first text-filtered expansion for any system fails hard and surfaces as an OperationOutcome exception.

Why CI did not catch it — semantic merge conflict

Neither PR could have caught this alone; it only breaks in combination.

PR what it did why it was green
#194 (feat/hts-ig-publisher-validate-code, merged 4df65385b) Added the code that uses concepts_search_fts, without the DDL Nothing in its test set reached the build path
#233 (fix/227-hts-expansion-collapse, merged e5320c5f2) Added expand_is_a_filter_with_text_filter_still_nests, which does reach it Its CI (run 29021783522) ran against a base that predated #194

Both merged green. The combination on main is broken.

Suggested fix

Add the missing table to crates/hts/src/backends/sqlite/schema.rs, mirroring concepts_fts. Inferred from usage (INSERT INTO concepts_search_fts(rowid, system_id, code, term), MATCH on the term, bm25() ranking, system_id used only as a post-filter):

CREATE VIRTUAL TABLE IF NOT EXISTS concepts_search_fts
USING fts5(system_id UNINDEXED, code UNINDEXED, term,
           tokenize='trigram case_sensitive 0');

Flagging rather than fixing: the tokenizer and which columns are UNINDEXED are my inference, not the author's intent — @sandhums should confirm the shape, since term carries both preferred displays and synonym designations.

Also worth considering: the startup path clears/prebuilds the other FTS tables; concepts_search_fts likely needs the same treatment.

Found while rebasing #261 onto main.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions