Fix $filter on searchable string fields (#27)#29
Merged
Conversation
Searchable Edm.String fields were indexed only as analyzed TextFields, so an exact TermQuery built from the filter literal never matched the lowercased, tokenized terms. Filterable/sortable/facetable searchable string fields now also emit a non-analyzed StringField under the same name, matching Azure's semantics where filters run on raw, non-analyzed content (https://learn.microsoft.com/azure/search/search-analyzers). Also: reject filters against fields with Filterable=false, and fix MergeDocument so dual-indexed fields aren't accidentally dropped during upsert. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #27.
Problem
When an
Edm.Stringfield wasSearchable=true(the default) and alsoFilterable=true,$filterqueries likeCategory eq 'Electronics'returned zero results. The field was indexed only as a LuceneTextField, so theStandardAnalyzerlowercased and tokenized the value at index time ("Electronics"→"electronics"), but the filter path built an exactTermQueryfrom the raw literal and never matched. The reporter's workaround — settingIsSearchable=false— worked only because it collapsed the field into theStringFieldbranch.Per Filters in Azure AI Search and Analyzers for text processing: filters run on raw, non-analyzed content, independent of whether the field is searchable.
Fix
SearchFieldExtensions.CreateField→CreateFields(now returnsIEnumerable<IIndexableField>). ForEdm.Stringfields that are searchable and filterable/sortable/facetable, index both aTextField(analyzed, for$search) and aStringField(raw, for$filter/$orderby/facets) under the same field name. Lucene merges postings by field name, soTermQuerymatches the raw side while analyzed-token search still works.UpsertIndexDocumentActionBase.MergeDocument: remove existing fields once per distinct name before re-adding, so dual-indexed fields aren't dropped during upsert.ODataQueryVisitor: addedEnsureFilterable(path)at the filter entry points — a filter referencing a field withFilterable=falsenow throws, matching Azure's behavior.Test plan
LuceneNetIndexSearcher_SearchableFilterableTestscovering (a)Searchable=true, Filterable=true, (b) schema defaults (exact repro of Searchable and Filterable fields return no results with filters applied #27), (c) rejection onFilterable=false.$searchstill matches tokenized terms on the dual-indexed field, and$filteron a non-filterable field errors.🤖 Generated with Claude Code