feat(validator): FHIR resource validation — schema engine, $validate, write-path enforcement#232
Draft
dougc95 wants to merge 9 commits into
Draft
feat(validator): FHIR resource validation — schema engine, $validate, write-path enforcement#232dougc95 wants to merge 9 commits into
dougc95 wants to merge 9 commits into
Conversation
…dation engine Phase 1 of the FHIR Schema based validator (see fhir-schema sibling repo): - FhirSchema serde model (differential, order-preserving, Arc-shared elements) - SchemaResolver trait + SchemaRegistry/CompositeResolver (SearchParameter registry pattern) - Cooperative schema-set walk: base layering, complex-type resolution, unknown-element, array/scalar shape, required/excluded, choices/choiceOf, fixed/pattern, numeric min/max, dynamic Resource resolution (Bundle.entry.resource, contained) - Deferred-effects collection for constraints/bindings (executed in a later phase) - Vendored upstream FHIR Schema conformance suite (MIT, provenance in UPSTREAM.md) with exact ordered error matching; fixtures 1,2,3,4,7 green, 5/6 gated until slicing lands Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Phase 2 — the full upstream conformance suite is now green (all 7 fixtures): - engine/slicing.rs: mark/sweep slicing with verbatim cardinality messages; closed/openAtEnd/ordered rules, @default absorption, and prohibited max:0 slices (all beyond the reference validator, pinned by extended fixtures) - extensions maps compile into synthetic {url}-matched slicing on the extension element at validation time - _field primitive-extension sidecars: base-element pairing, shape mirroring, Element-part validation with null gaps - elementReference resolution for recursive structures (Questionnaire.item) - extended fixture suite pinning Helios-specific semantics (excluded, numeric min/max, fixed/pattern wording, slicing rules, sidecars, elementReference, required-satisfied-by-choice-branch) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…idator-cli
Phase 3:
- converter/: StructureDefinition (raw JSON, snapshot OR differential) to
FhirSchema — ED-id tree reconstruction, choice [x] expansion,
discriminator-to-pattern-match translation, extension-slicing to
extensions sugar, contentReference to elementReference, targetProfile to
refers, min/max to required/excluded/cardinality, ele-1/ext-1 dedup,
primitive value-regex extraction. Golden tests pin every mapping.
- generate-schema-packs bin (gen-packs feature): converts
crates/fhir-gen/resources/{V}/profiles-{types,resources,others}.json.
All four versions convert with ZERO failures: R4 257, R4B 247, R5 296,
R6 228 schemas; packs total ~437 KB gzipped, committed under packs/.
- packs.rs: include_bytes + per-version OnceLock core registries
(profiles-first insertion so core names win aliases).
- pack_smoke (#[ignore]): real-world Patient with choice types, sidecars,
contained resources and extensions validates clean against the converted
R4 core; structural breakage and Bundle dynamic resolution verified.
- validator-cli bin (cli feature): file/stdin, --fhir-version, --profile,
--profile-file (converts SDs on the fly), text/json output, exit 0/1/2.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…aints, terminology bindings Phase 4: - engine/primitives.rs: JSON type-class checks from the primitive's declared type + anchored spec-regex checks (process-wide compiled-regex cache); kind 'primitive-value'. Upstream fixtures declare bare primitives, so the exact-match conformance contract is untouched (pinned by extended/primitives.json). - effects.rs: execute() pipeline — constraints deduped by (path,id) across cooperative layers, severity mapping (error/warning; guideline skipped), suppress list, not-evaluable surfaced as warnings; bindings filtered to required strength, coded shape from declared type with value-shape fallback, fail-open/fail-closed terminology outages. Async TerminologyProvider (async-trait), sync ConstraintEvaluator with batch evaluate_all. - Validator::validate(): sync walk + effects execution; validate_sync remains pure. - fhirpath_effects.rs (feature 'fhirpath'): typed-parse-once per resource, focus via 'path.all(expr)' wrapping, FHIR invariant semantics (empty → pass; the JS reference fails empty — known bug there), Arc-cached expression ASTs. - 6 stub-driven effects tests + real-invariant test (pat-1 from the R4 pack; binary carries an MSVC 14.44 LNK1318 caveat, scenarios also covered via the rest integration suite). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Phase 5:
- routes: POST /{type}/$validate and GET|POST /{type}/{id}/$validate
(before the catch-all; GET validates the stored resource) — the
CapabilityStatement's long-advertised validate operation is now real.
- handlers/validate.rs: raw-resource or Parameters bodies (resource/mode/
profile parts, query fallbacks), mode=create|update|delete|profile
semantics, always-200-OperationOutcome contract (4xx only for malformed
requests).
- validation.rs: ValidationService (embedded core packs per version,
FHIRPath constraint evaluator with dom-6 suppressed by default,
meta.profile honored, pluggable TerminologyProvider) + ErrorKind →
IssueType mapping with bracket-indexed FHIRPath expressions.
- OperationOutcome: IssueType gains Invariant and CodeInvalid.
- AppState carries the service (always present; write-path gating comes
with HFS_VALIDATION_* in the next phase).
- 12 integration tests incl. real spec invariant (pat-1) evaluation
through the full HTTP stack.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…g, tenant profiles Phase 6: - ValidationConfig (HFS_VALIDATION_MODE off|log|enforce, META_PROFILES, UNKNOWN_PROFILE, CONSTRAINTS, SUPPRESS_CONSTRAINTS, TERMINOLOGY off|remote, TERMINOLOGY_TIMEOUT_MS, TERMINOLOGY_FAIL open|closed, STORED_PROFILES) following the BulkExportConfig pattern; remote terminology requires HFS_TERMINOLOGY_SERVER (validated). - ValidationService::from_config + check_write: enforce mode rejects invalid writes with 422 carrying the full multi-issue OperationOutcome (new RestError::ValidationFailed); log mode traces and proceeds. - Hooks on every write path: create, update, conditional create/update, batch (per-entry 422 results), and transaction (atomic pre-execution rejection). - Per-tenant profile registries: stored StructureDefinitions convert and register on write (all write paths), overlay the core packs via CompositeResolver for $validate, meta.profile, and enforcement. In-memory since process start (startup warm-load is future work). - RemoteTerminologyProvider: ValueSet/$validate-code against HFS_TERMINOLOGY_SERVER with a 300s verdict cache. - Docs: config env table, run-hfs-server skill section, CLAUDE.md workspace table (validator + web crates). - 6 enforcement integration tests (enforce/log/off matrix, batch per-entry, stored-profile $validate round-trip, meta.profile enforcement); rest and batch conformance suites still green (39 + 57). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…latency smoke Phase 7 (quick wins; remaining hardening documented in lib.rs): - fuzz_lite: 2500 deterministic mutations of a rich Patient plus pure-junk roots — the walk never panics on arbitrary JSON. - pack_smoke: every enabled version's embedded pack (R4/R4B/R5/R6) loads, resolves core schemas, validates a Patient, and catches unknown elements. - Latency smoke: ~300µs per typical Patient structural validation in DEBUG builds — the 5ms release-plan target is beaten by an order of magnitude. - lib.rs 'Current limitations' section pins the hardening backlog: type/profile/binding/resolve-ref slice matchers, refers enforcement, extensible-binding warnings, schema-set memoization, hts in-process terminology adapter, core extension-definitions vendoring. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Resolve conflicts: - crates/rest/Cargo.toml: keep both helios-fhir-validator and helios-observability deps - handlers/mod.rs: keep both user_settings and validate modules + re-exports - handlers/batch.rs: combine our StructureDefinition profile-upsert with main's entry_error handling - error.rs: ValidationFailed surfaces its full OperationOutcome via into_response; add a fallback triple in client_response - Cargo.lock: regenerated Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
# Conflicts: # crates/rest/src/routing/fhir_routes.rs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds FHIR resource validation to HFS: a new
helios-fhir-validatorcrate (a runtime FHIR Schema engine), the$validateoperation, and optional write-path enforcement driven byHFS_VALIDATION_*config — across R4 / R4B / R5 / R6.70 files changed, ~8.9k insertions.
Relationship to the Validation strategy discussion
This work is a response to the architecture strategy document in discussions/215 — "Validation". It delivers most of the functional scope that discussion commits to, but takes a deliberately different implementation path, so it's offered for review/discussion rather than as a literal implementation of that doc.
Where this PR agrees with #215:
helios-fhirpath(not re-implemented) — exactly as the doc argues for.OperationOutcomeas the public contract; per-version R4/R4B/R5/R6 dispatch.$validateoperation + write-path validation config surface.Where this PR diverges from #215 (worth a maintainer decision):
&'staticRust tables** via extendinghelios-fhir-gen+ a new#[derive(FhirValidate)]macro inhelios-fhir-macrohelios-fhir-validatorcrate with a runtime engine that loads embedded compressed schema packs and interprets them (mirrors thefhir-schemareference impl)helios-validationhelios-fhir-validatorhelios-hts$validate-codefallback$validate-codeTerminologyProvider(no bundled local tables, no HTS wiring yet).fhirschema.jsonartifacts per version*.json.gzblobs (regenerated by thegenerate-schema-packsbin)Aspect coverage (against #215's aspects table)
helios-fhirpathmeta.profile+ stored/tenant profiles, slicing, fixed/pattern, choice typesReference.typetarget profiles)refers; target-profile checking is a follow-upOperational surface
$validate— accepts a raw resource or aParameterswrapper (resource/mode/profile), honors?profile=and?mode=, supportscreate/update/delete/profilemodes, and always returns200 OK+OperationOutcome(non-200 only for malformed calls, e.g.mode=profilewith no profile).mode=deletedoes no content validation.HFS_VALIDATION_MODE = off | log | enforce;enforcerejects non-conformantcreate/updatewith422 Unprocessable Entity+OperationOutcome. Off by default.HFS_VALIDATION_MODE,HFS_VALIDATION_UNKNOWN_PROFILE(error|warn|ignore),HFS_VALIDATION_TERMINOLOGY/_TERMINOLOGY_FAIL/_TERMINOLOGY_TIMEOUT_MS,HFS_VALIDATION_CONSTRAINTS/_SUPPRESS_CONSTRAINTS,HFS_VALIDATION_META_PROFILES,HFS_VALIDATION_STORED_PROFILES.validator-cli(clifeature) for standalone resource validation.Testing
tests/conformance.rs) against vendored upstream FHIR Schema fixtures (tests/fixtures/upstream/).Not in this PR (follow-ups from #215)
#[derive(FhirValidate)]/helios-fhir-gencodegen path (this PR uses the runtime-engine alternative instead).helios-htsfallback.$validatenightly + conformance badge.type/profileslice discriminators (per Validation #215, staged).🤖 Generated with Claude Code