Store SearchParameters in storage; make the in-memory registry a TTL cache over it

[smunini]

Summary

SearchParameters are currently loaded from disk into an in-memory registry and never written to storage. This makes the in-memory registry the sole source of truth, which breaks two things: clients can't discover the server's search parameters via the FHIR API, and a SearchParameter POSTed to one node in a cluster is invisible to every other node.

We should invert the relationship: storage becomes the source of truth, and the in-memory registry becomes a cache that reads from storage. A new env-var-configurable TTL controls how long a node may serve a stale cache before re-reading.

Current behavior

At startup, SearchParameterLoader reads data/search-parameters-{r4,r4b,r5,r6}.json into an Arc<RwLock<SearchParameterRegistry>>. On a fresh SQLite backend the boot log reports:

SearchParameter registry initialized: 1377 total (1368 spec from ./data/search-parameters-r4.json, 9 fallback, 0 custom) covering 136 resource types

Nothing in crates/hfs or crates/persistence ever writes those parameters into a storage backend. The result is two disconnected sets of search parameters:

1. The registry — what the server actually uses to resolve searches. In-memory only.
2. Stored SearchParameter resources — what GET /SearchParameter queries. Empty until a client POSTs one.

This was confirmed empirically against a fresh SQLite backend:

$ GET /SearchParameter?base=Patient
200
{"resourceType":"Bundle","type":"searchset","entry":[],"total":null}

The request is well-formed and the search executes — base is a registered token param (SearchParameter.base), and SearchParameter resolves through the generic resource routes at crates/rest/src/routing/fhir_routes.rs:288. Nothing is rejected; there is simply nothing in storage to match. POSTing a single custom SearchParameter with base: ["Patient"] returns 201, and the same query then returns exactly that one entry — not the ~1300 registry params. So the endpoint is a real, working FHIR search over stored resources; only the seeding is absent.

Today the sole client-visible view of the registry is GET /metadata → rest[0].resource[?type=='Patient'].searchParam, built by build_search_params (crates/rest/src/handlers/capabilities.rs:407) from registry.get_active_params(resource_type).

Proposed change

1. Write SearchParameters to storage first

On startup, seed the storage backend from SearchParameterLoader rather than loading straight into memory. Seeding must be idempotent and safe when multiple nodes boot concurrently against a shared database — an existing parameter with the same canonical URL should not be duplicated or clobbered.

2. In-memory registry reads from storage

The registry becomes a read-through cache over storage rather than over disk. Loading spec files stays as the seeding path; resolving searches reads what storage holds, so custom SearchParameters POSTed by clients participate in search resolution the same way spec ones do.

3. New cache TTL setting

Introduce HFS_SEARCH_PARAM_CACHE_TTL (naming follows the existing HFS_<AREA>_<SETTING> convention, cf. HFS_BULK_EXPORT_CLEANUP_INTERVAL).

Default: 1 hour.

Semantics in a clustered deployment: when a client POSTs a new SearchParameter, the receiving node performs the write to storage. Other nodes in the cluster will not observe the new parameter until they reach their cache check limit, at which point they re-read and refresh their registry from the database. This is deliberate eventual consistency — the high default trades propagation latency for avoiding a database read on every search.

Notes / open questions

• Behavior when a node's re-read fails (stale-serve vs. hard-fail) should be decided explicitly rather than falling out of the implementation.
• /metadata is built from the registry, so a node's advertised CapabilityStatement will lag a cluster-mate's by up to the TTL. Worth confirming that's acceptable.
• Interaction with the search-parameter extractor matters: indexing depends on the registry, so a parameter that appears mid-TTL won't have indexed pre-existing resources. Whether adding a SearchParameter triggers reindexing is out of scope here but should not be left implicit.
• Related prior context: register() rejects duplicate URLs, but params_by_type uses HashMap::insert() — the same code under a different URL silently overrides the lookup. Seeding from storage should not make that easier to hit.

[smunini]

Heads-up: #239 is a registry bug that this issue's design should account for before it lands.

SearchParameterRegistry::register() guards duplicate canonical URLs, but params_by_type is keyed by (base, code) and written with .insert() (registry.rs:294 vs :311). A parameter with a new URL but an existing (base, code) registers Ok(()) and silently overwrites whichever definition previously answered that query — no error, no log, len() unchanged.

Once storage is the source of truth, every POST /SearchParameter flows into that same register() path with an arbitrary URL, so each one becomes a chance to silently shadow a spec parameter. #239 also documents two follow-on bugs (unregister() evicting the wrong param, update_status() flipping which definition resolves), all three reproduced with passing tests.

The open question #239 raises is one this issue probably has to answer anyway: is a stored param overriding a spec param on (base, code) a supported override or a conflict? That determines whether SearchParameterSource needs a documented precedence order or register() needs to reject the collision outright.

[angela-helios]

Implementation is up in #244, stacked on #242 (the shadowing fix this depends on). Note on linking: GitHub only activates Closes #… keywords for PRs based on the default branch, so the Development link will attach automatically when #242 merges and #244 retargets to main — the closing keyword is already in the PR body.