feat(search): natural-language search — translate to a query, never run it (#255)

[angela-helios]

Closes #255.

Stacked PR. Base is feat/237-238-registry-compartment-ui (#252 → #241 → main), because the Search page reuses the Search Builder, the resource rail and the results table from those branches rather than duplicating them. Merge the chain bottom-up.

Type "female patients over 65 with a diabetes diagnosis" and get a FHIR search query you can read, correct, and run.

The design decision everything else follows from

POST /$nl-search returns a query, not results. The client then runs that query through the ordinary FHIR search path.

That keeps auth, tenancy, audit and search semantics on the one code path we already trust — the translator has no privileges of its own and never touches storage — and it makes the query reviewable before it runs. Nothing is translated silently.

So the UI shows the generated query in an editable strip, and Run is always a separate click (open question 2 on the issue: show-first-then-click, at least for the first iteration).

Nothing the model says is trusted

Three layers, and the prompt is deliberately the weakest of them:

1. The output shape is forced, not requested. The model answers through a single strict tool call. A jailbroken model has nowhere to put free text.
2. The query is validated server-side. Every parameter is checked against the SearchParameter registry, and the resource type against the FHIR version. Unknown parameter, unknown type, smuggled payload → fails closed with a friendly error, never executed.
3. The prompt (crates/rest/src/handlers/nl_search_prompt.md, its own reviewable file) scopes the model to translating search intent, treats the input as data rather than instructions, and answers supported: false to anything else.

The prompt is built from the server's real searchable vocabulary out of the registry, so it can never advertise a parameter this server cannot execute.

The two behaviors you asked for

Ask for something out of scope and it refuses rather than guessing:

$ curl -sX POST /$nl-search -d '{"text":"write me a bubble sort algorithm in python"}'
{"supported":false,"reason":"That is a request to write code, not a search over this server's data."}

Pack a real search and something else into one paragraph, and it answers only the part that is in scope — and says so:

$ curl -sX POST /$nl-search -d '{"text":"find patients named smith, and also write a poem about FHIR"}'
{
  "supported": true,
  "target": "Patient",
  "query": "name:contains=smith",
  "explanation": "Patients whose name contains Smith.",
  "caveats": ["Only the first request was translated; the rest of the input was not a search and was ignored."]
}

Protecting the operator's key

An exposed natural-language box is an exposed LLM endpoint someone else is paying for.

• Sliding-window rate limit, a daily ceiling (the slow-drip backstop), and an input length cap.
• Keyed per authenticated user within a tenant, falling back to the peer address when auth is off — new PeerIp extractor, and the binary now serves with ConnectInfo. With neither, one bucket per tenant: the most restrictive reading, deliberately.
• Over the limit → 429 + OperationOutcome (throttled). The limiter runs before the provider call, so a throttled request costs nothing.

Three states, and off means gone

HFS_NL_SEARCH_ENABLED	API key	Result
false	anything	Endpoint 404s, /ui/search is never routed, the sidebar entry stays the coming-soon placeholder. No mention anywhere.
true	unset	The page advertises the feature, names the env vars, links the how-to. The visual builder still works.
true	set	Working.

Screenshots

Translated — the query is shown, editable, and not yet run. Explanation and caveats underneath.

Run it, and the results come back through the normal search path.

Off-topic is refused — and the previous query in the strip is left untouched.

Enabled but unconfigured: the setup state. An operator who has never heard of the feature can turn it on without leaving the page.

Design is Brett's Figma "Search V1.0" frame (node 145:348): the segmented Natural language / Visual builder switch over one QUERY strip. One deliberate deviation — Figma runs the search straight from the natural-language box, but the issue requires the query be seen and correctable first, so the strip is editable and Run is explicit.

How I verified it

No provider key on hand, so I stood up a mock Anthropic Messages endpoint and pointed HFS_NL_SEARCH_BASE_URL at it — which also exercises the self-hosted/proxy escape hatch. End to end against a running server with seeded patients:

• "female patients over 65 with a diabetes diagnosis" → Patient?gender=female&birthdate=le1961-07-14&_sort=-birthdate, which returns exactly the 5 women born on or before that date (the male patient correctly excluded), newest first.
• The mock's logs confirm the system prompt reaching it is 38 KB of this server's registry vocabulary, not a hardcoded list.
• 11 rapid calls → 200s then 429 once the window filled.
• ENABLED=false → /ui/search and /$nl-search both 404, and zero links to the page in the rendered sidebar.
• Browser check on both /ui/queries and /ui/search after the shared-partial refactor: builder syncs, results render, no JS errors.

Tests: 6 HTTP integration tests (crates/rest/tests/nl_search.rs), 11 unit tests over validation / prompt / rate-limit keying, 3 new UI state tests, locale parity across en/es/de. cargo fmt and clippy clean.

Notes for review

• The live model has never been called. Everything above ran against a mock. The refusal and packed-request behavior is what the prompt instructs and what the server enforces, but nobody has watched a real model do it yet — worth one run with a real key before we call this done.
• book/src/SUMMARY.md is stale. It still lists only the original ch01…ch14 scaffold; components/, configuration/ and getting-started/ are all orphaned from it, so mdBook isn't building them today. I added my page to SUMMARY so it is reachable, but that tree needs a separate fix.
• The results table shows id + updated only — it derives columns from _elements. Figma shows NAME / GENDER / BIRTH DATE; that needs a column model and belongs in its own issue (it's the existing renderer, shared with Saved Queries).
• Default provider is Anthropic, default model claude-opus-4-8, with HFS_NL_SEARCH_BASE_URL as the escape hatch (open question 1: one provider to start).

[angela-helios]

Now verified against a real model — and it found two bugs the mock never could

Angela ran this against the live Anthropic API. Everything works, in English and Spanish, including the two behaviors the issue asks for. But the first real run surfaced a bug I would never have caught with a mock:

The model has no clock. Asked for "female patients over 65", it computed the cutoff against a date it invented — 17 months in the past — and returned birthdate=le1960-02-14, complete with a confident caveat stating the wrong assumption. My prompt told it to "compute from today's date" and then never gave it today's date.

Fixed: the prompt now carries the date and is told never to guess it. Same request now:

Patient?gender=female&birthdate=le1961-07-14&name:contains=ana

A malformed answer was being echoed into the browser. One run had the model leak its own tool-call markup into resource_type, and my error message interpolated that straight into text rendered in the UI. The validation layer did its job — the search never ran — but untrusted model output has no business leaving the server. Now the caller gets a fixed sentence and the raw value goes to the log. Regression test added.

That leak was transient, so an unusable translation is now retried once (two provider calls, hard maximum) rather than surfacing as an error. Also dropped "strict": true from the tool definition — an OpenAI field the Anthropic Messages API doesn't have.

The three cases, against the real model

>>> female patient over 65 years old ana bubble sort algothm in java
  query:   Patient?gender=female&birthdate=le1961-07-14&name:contains=ana
  caveat:  Ignored the trailing request about a bubble sort algorithm in Java,
           which is not a FHIR search.

>>> escribeme un algoritmo de bubble sort en java
  RECHAZADO: Esta solicitud es para generar código y no es una búsqueda FHIR
             contra este servidor.

>>> pacientes mujeres mayores de 65 anios con diagnostico de diabetes
  query:   Patient?gender=female&birthdate=le1961-07-14&_has:Condition:patient:code=44054006
  caveat:  'Diabetes' mapped to SNOMED CT 44054006 (Diabetes mellitus type 2);
           if you meant a different code or ICD-10, specify it.
  caveat:  Does not restrict by clinical status; add clinical-status if you only
           want active diagnoses.

It refuses in the language it was asked in, and packed requests get the search translated and the rest explicitly dropped in a caveat.

One thing reviewers should know: the prose the model writes can still be sloppy where the query is right — in the first case its explanation said "born on or before 2026-07-14" while emitting the correct le1961-07-14. The query is what executes and it is validated; the explanation is commentary. Worth watching, not worth blocking on.

[angela-helios]

Visual builder — the conditions are fed by the real SearchParameter registry

The Visual builder tab (beside Natural language) in action, on Patient:

Conditions, includes and result controls parse two-way from the editable QUERY strip. The condition parameter field autocompletes from a <datalist> served at /ui/queries/params?type=Patient, which is built from the actual SearchParameter registry, filtered to the parameters that apply to the resource type. Here are the 32 real Patient parameters it offers (rendered visibly for the shot, since a native datalist popup doesn't capture headless):

They are genuine, resource-specific FHIR search parameters — active, address-city, birthdate, death-date, general-practitioner, identifier, organization, telecom, plus the Resource-level _id/_profile/_lastUpdated — and you will not see parameters from other resource types (no clinical-status, no code). A test asserts exactly this (queries_param_catalog_is_a_registry_fed_fragment).

One honest gap, for the record: the condition datalist suggests (it is a non-enforcing <input list>), and the _include/_revinclude value is free text (Patient:organization) — the two include keywords are real, but the builder does not yet enumerate which reference params are valid include targets for the resource. The registry already has that information (the reference-type params and their target types), so wiring it is a small follow-up; flagged, not silently missing.

[angela-helios]

Visual builder filtering + listing results, and the configured NL state

Visual builder — filter and list the matched patients. Building gender=female, _sort=-birthdate, _count=6, _elements=name,gender,birthDate and hitting Run executes the query through the ordinary FHIR search path and lists the results in-page, with columns driven by _elements:

Six female patients, newest birth date first, rendered with Name / Gender / Birth date — the results come straight from the FHIR API (no LLM involved on the builder path).

Configured NL state — no setup / how-to card. With HFS_NL_SEARCH_API_KEY set, the page shows the working natural-language input (and the mode toggle) instead of the "how to enable it" card:

The describe-box, example chips, and the editable QUERY strip are all present; the setup instructions only appear when no key is configured.