design review: Rust-native evidence-first OCR engine

[Sam123ben]

Parent: #1. Related reviews: #3, #4, #6 and #7.

Direction

Build a Tesserix-owned OCR engine and production runtime in Rust. Established external OCR systems are benchmark competitors only; we do not adopt or copy another project’s runtime, pipeline, source or public contract.

“Our OCR” means Tesserix owns preprocessing, inference pipeline, model contract, deployed artefacts, normalization, evidence, calibration, evaluation and release gates. Training experiments may use Python; production inference remains Rust with signed exported model artefacts.

Product niche

Agent-grade, evidence-first document OCR:

• immutable original-page evidence for every recognized value;
• calibrated uncertainty and explicit unusable, unknown and partial outcomes;
• forms, receipts, invoices, statements, IDs, contracts and mobile scans;
• deterministic validation and human-review routing;
• untrusted-content and prompt-injection signals for agents;
• streaming pages, bounded resources and private/self-hosted execution.

Proposed runtime

• Rust stable, Tokio, Axum/Tower, SQLx, thiserror, serde and tracing/OTel.
• ONNX Runtime through a pinned Rust integration as the initial CPU/CUDA/TensorRT execution layer.
• Separate crates for domain, image transforms, detection, recognition, layout, runtime, evaluation and service.
• Sandboxed decode/inference workers because codecs, GPU drivers and ONNX Runtime remain native/unsafe dependencies.
• Models are warmed and pooled per device/version, batched by bounded shape/pixel budgets and promoted independently.

Required design and benchmark gate

• Approve the first narrow cohort: printed English scans/mobile images plus source geometry and reading order.
• Select legally usable bootstrap detector/recognizer weights after model-license review.
• Define signed model manifest, tensor contracts, calibration and rollback predecessor.
• Build identical-hardware baselines against established open-source and approved cloud OCR.
• Measure CER/WER, evidence IoU, reading order, layout/table cohorts, p50/p95/p99, pages/s, CPU/GPU/memory and cost.
• Prove warmed sessions, bounded batching, page streaming and memory ceilings.
• Prove transforms map all evidence back to immutable original-page coordinates.
• Fuzz decoders, tensor validation, geometry and post-processing.
• Prove raw text and field values never enter logs/traces.
• Keep VLM use limited to bounded low-confidence regions with separate provenance.

Temporal risk gate

Temporal Rust SDK v0.8.0 is Public Preview/pre-release as of 2026-09-02. Before production selection, prove replay, worker versioning, cancellation, signals, child workflows, heartbeats, 300-page history bounds, crash recovery and a 24-hour upgrade soak. If it fails, use a narrow stable Go Temporal runner calling the Rust service; do not build a custom durable state machine merely for language purity.

Approval

• Rust/service owner
• OCR/ML owner
• Model licensing/data-governance owner
• Temporal/platform owner
• Security/native supply-chain owner
• SRE approves measured benefit, cost and failure isolation

Implementation #8 remains blocked until this review records the selected first profile, model artefacts, benchmark thresholds and workflow runtime.

[Sam123ben]

ADR-0004: Rust-native, evidence-first OCR engine

Status: proposed

Context

The platform needs OCR that is fast, private, auditable and safe for AI agents. Existing OCR projects are useful references and benchmark competitors, but adopting one as the runtime would make its pipeline, output model, release cycle and deployment constraints part of our product boundary.

The valuable product is not raw text alone. It is a reproducible document observation graph: original-page geometry, reading order, calibrated confidence, transform provenance, structured layout, validation and review decisions. That is where Tesserix can build differentiated, reusable capability.

Decision

Build a Tesserix-owned OCR engine and production runtime in Rust. External OCR implementations are benchmark material only; the service will not adopt or copy another project's runtime, pipeline, source or public contract.

The production engine owns:

• bounded document/page decoding and preprocessing;
• image-quality measurement and transform provenance;
• text-region detection, orientation and recognition inference;
• layout, table and reading-order reconstruction;
• confidence calibration and evidence coordinates;
• provider/model-independent observations;
• batch scheduling, streaming and hardware execution profiles;
• normalization, deterministic validation and review routing.

Model training and experimentation may use Python because the training ecosystem is materially stronger. Exported, signed and benchmarked model artefacts cross a strict model contract into Rust production inference. Production serving does not embed Python.

Start with ONNX Runtime through a pinned Rust integration because it provides mature CPU, CUDA and TensorRT execution providers. Keep inference behind an internal engine trait so a model can move to Candle, tract or a custom runtime only after a measured reason exists. The trait abstracts Tesserix model stages, not arbitrary third-party providers.

Product niche

The engine is optimized for agent-consumable business documents rather than generic scene text:

• scanned and mobile-captured documents;
• forms, receipts, invoices, statements, IDs and contracts;
• evidence for every extracted claim;
• explicit unusable, unknown and partial outcomes;
• multilingual and mixed-script expansion driven by measured tenant demand;
• untrusted-content marking and prompt-injection signals;
• deterministic financial, date and identity validation;
• human correction as versioned provenance, not silent training data.

Model ownership progression

1. Use legally approved open model weights as replaceable bootstrap artefacts where they meet the model contract.
2. Establish Tesserix golden datasets, calibration and deployment benchmarks before fine-tuning.
3. Fine-tune detector, recognizer and layout models for the chosen document and language cohorts.
4. Train or distill models only where owned data and measurements show a defensible quality, latency, privacy or cost advantage.

“Our OCR” means we own the pipeline, contracts, training/evaluation policy, deployed artefacts and release gates. It does not require inventing a foundation architecture before the first useful result.

Runtime structure

Use a Cargo workspace with narrow crates: ocr-domain, ocr-image, ocr-detect, ocr-recognize, ocr-layout, ocr-runtime, ocr-eval, and the document-service binary. Crates communicate through typed observations and immutable page artefacts, not JSON internally.

CPU work runs outside Tokio executor threads in bounded worker pools. Model sessions are loaded once, warmed, pooled per device, model and version, and scheduled by pixel budget rather than document count. Inputs are grouped into bounded shape buckets for batching. Large documents stream page observations and never require the entire result in memory.

Native PDF/image codecs and ONNX Runtime remain unsafe/native dependencies even behind Rust. They execute in a sandboxed worker with no network, read-only filesystem, strict CPU, memory, time and output bounds, and disposable process isolation.

AI boundary

A vision-language model is not the default OCR engine. It may interpret a bounded low-confidence region after deterministic OCR, with the original region and prompt-injection controls preserved. Semantic extraction and agent reasoning remain downstream stages and cannot modify OCR evidence.

Temporal qualification

Temporal's Rust SDK is Public Preview at pre-release v0.8.0 as of 2026-09-02. Development may spike it, but production selection requires replay, versioning, cancellation, heartbeat, child-workflow, 300-page history and crash/upgrade soak evidence. If it fails, use a narrow stable Go Temporal workflow runner calling the Rust service through a versioned internal protocol. Do not build a custom durable state machine for language purity.

Alternatives

• Adopt an external open-source OCR runtime: rejected because its pipeline, compatibility and release constraints would become part of our product boundary.
• Pure Python service: rejected for the production runtime because predictable concurrency, memory bounds and native service safety are core requirements.
• Train every model from scratch immediately: rejected because it delays evaluation and product learning without proving an advantage.
• Vision/VLM for every page: rejected because cost, latency, reproducibility and prompt-injection surface are unacceptable.

Consequences

Tesserix owns a coherent OCR product and can optimize it for agents. It also owns model compatibility, preprocessing correctness, calibration, multilingual expansion and hardware qualification. The first release must therefore be deliberately narrow and benchmarked rather than claiming universal OCR quality.

[Sam123ben]

Rust OCR engine design

Product thesis

Build the best evidence-first OCR engine for AI agents and document workflows, not the broadest demo. Every recognized token must be traceable to an immutable source region, carry calibrated uncertainty and remain explicitly untrusted when consumed by an agent.

The initial design point remains 20 submitted jobs/s and 100 pages/s peak, with typical four-page documents, a 300-page/100 MiB hard ceiling and page streaming for long inputs. The API p99 target is under 300 ms; eligible two-page interactive processing targets p95 under 10 seconds after benchmark calibration.

Engine pipeline

flowchart LR
    Input[Immutable page] --> Decode[Bounded decode]
    Decode --> Quality[Quality vector]
    Quality --> Geometry[Rotate, crop, deskew, dewarp]
    Geometry --> Detect[Text-region detector]
    Detect --> Orient[Region orientation]
    Orient --> Recognize[Text recognizer]
    Recognize --> Layout[Layout and reading order]
    Layout --> Calibrate[Confidence calibration]
    Calibrate --> Evidence[Observation graph and evidence]
    Evidence --> Validate[Deterministic validation]
    Validate --> Output[Page stream / result / review]
    Calibrate -->|bounded low-confidence region| Vision[Optional vision interpretation]
    Vision --> Evidence

Loading

Each stage consumes and produces typed, versioned artefacts. The original page is immutable. Geometry operations append an invertible transform so every derived polygon maps back to the original page.

Cargo workspace

Crate	Responsibility
ocr-domain	Newtyped IDs, page geometry, observations, confidence, language/script, errors and version contracts
ocr-image	Bounded decode, quality vector, rotation, crop, deskew, dewarp, denoise and transform graph
ocr-detect	Detector model contract, tiled inference, polygon post-processing and region filtering
ocr-recognize	Orientation, script/language routing, recognition batching and token/character alternatives
ocr-layout	Reading order, blocks, headings, lists, tables, cells, selection marks and page sections
ocr-runtime	Signed model loading, warmed sessions, device pools, shape buckets, pixel budgets and streaming execution
ocr-eval	Golden manifests, CER/WER, geometry/table/layout metrics, calibration and benchmark reports
document-service	Axum API, authentication, tenancy, SQLx metadata/outbox, workflow integration and OTel

Crates use Rust stable edition 2021, Tokio for I/O, Axum/Tower for HTTP, SQLx for Postgres, serde for edges, thiserror for typed failures and tracing/OTel for telemetry. CPU inference and preprocessing never block Tokio executor threads.

Model contract

Every deployed model artefact has a signed manifest containing:

• model family, semantic version and immutable digest;
• stage and compatible input/output schema versions;
• supported scripts/languages, image shapes and precision;
• training/evaluation dataset provenance permitted for release;
• license and redistribution obligations;
• runtime and execution-provider compatibility;
• calibration version and approved cohorts;
• measured accuracy, latency, memory and cost envelope;
• rollback predecessor.

The runtime refuses unsigned, incompatible or unapproved artefacts. Model outputs are untrusted numeric tensors until stage-specific validation establishes shape, bounds and finite values.

Inference runtime

Initial inference uses ONNX Runtime through a pinned Rust binding. CPU, CUDA and TensorRT execution profiles are separate release artefacts because numeric precision and operator support can change results.

Performance comes from:

• one warmed session pool per device, model and version;
• bounded dynamic batches grouped by image-width and shape bucket;
• pixel-budget admission rather than file-count admission;
• detector tiling only above a measured resolution threshold;
• recognition batching across compatible regions and pages;
• reusable aligned image and tensor buffers with explicit memory ceilings;
• page-level pipeline overlap and streaming results;
• content, model and profile-addressed caching with tenant scope;
• optional quantization and distillation only after cohort regression checks.

No request may select an arbitrary model path, operator library, device, YAML or execution-provider option.

First model profile

The first production profile is intentionally narrow:

• printed English text;
• clean scans and mobile photographs;
• PNG, JPEG and WebP plus rendered PDF pages;
• text regions, line and word text, confidence, original-page polygons and reading order;
• CPU baseline and one approved NVIDIA GPU profile.

Tables, handwriting, mixed scripts, formulas and complex layouts are added as separately evaluated profiles. Hindi and other Indian scripts, Australian invoices and receipts, and product-specific forms are strong candidate cohorts, but their priority requires real tenant samples and approved labels.

Confidence and evidence

Detector, recognizer, layout and quality scores remain separate. Calibration is measured per model version, script, document class, acquisition type and quality band. Unknown calibration is not zero and cannot auto-accept a critical field.

The observation graph stores token, line and block relationships, alternatives where useful, original and derived polygons, transform chain, reading-order edges, model/profile versions and calibration cohort. Downstream extraction cites observation IDs rather than copying unverifiable text.

Safety and failure isolation

Rust reduces memory-safety risk in owned code but does not make PDF codecs, image libraries, GPU drivers or ONNX Runtime safe. Decode and inference workers are disposable sandboxes with no network, no shell, read-only root filesystem, generated work paths, strict input, output, time, CPU and memory limits, and narrow object access.

NaN and infinite tensors, impossible coordinates, invalid UTF-8, output explosions, model mismatch and GPU faults are typed failures. A page failure cannot terminate the API or restart successful pages. Raw document text never enters logs or trace attributes.

AI-assisted capability

AI helps in four controlled places:

1. label assistance with human approval for golden datasets;
2. hard-example mining from consented, reviewed corrections;
3. training, distillation and calibration experiments;
4. bounded interpretation of low-confidence regions with page evidence.

An AI result cannot overwrite deterministic OCR evidence. It is an additional observation with its own model version, confidence, cost and provenance. Agent reasoning remains outside the OCR engine.

Benchmark and promotion

Established open-source OCR implementations, cloud OCR providers and relevant open models are benchmark competitors, not runtime dependencies. Every candidate runs on identical images, hardware and cohorts.

Promotion requires no regression beyond approved tolerance in CER, WER, evidence IoU, reading order, tables and layout, critical-field exact match, unsafe auto-accept, p95 and p99 latency, peak memory and cost per successful page. Reports include cold and warm runs and never combine unknown cost with zero.

Delivery slices

1. ocr-domain plus golden and evaluation contracts and fixture renderer.
2. Safe Rust page decode, transform graph and quality vector.
3. ONNX detector with typed polygons and deterministic post-processing.
4. Recognizer batching with token alternatives and calibrated confidence.
5. Reading order and provider-independent page stream.
6. API, workflow and result-store integration and failure recovery.
7. Layout, table and multilingual profiles driven by cohort evidence.
8. Agent tool and DevAI evaluation only after service gates pass.

The first usable demonstration must process a small approved corpus end-to-end and show source highlights, latency, memory, CER/WER and trace output. A screenshot of plausible text is not an acceptance test.

[Sam123ben]

Implementation direction confirmed: the service and OCR inference runtime are Rust. The OCR agent remains a separately versioned tesserix/ai-agents artifact using Tesserix ADK 0.54.0 once the reviewed release is published. The Rust service exposes provider-neutral APIs/events; the agent consumes them through the Australis tool and is evaluated in DevAI. This keeps model inference, semantic extraction, agent reasoning and product policy independently measurable and releasable.

[Sam123ben]

Agent compatibility gate update (2026-09-03): the OCR agent must use the immutable published Tesserix ADK v0.54.0 release. tesserix/agent-development-kit#311 is green but the tag/release does not yet exist, so no consumer will pin a branch, commit masquerading as a release, or floating container tag. After review/merge/release, the agent build must pin the package and base-image digest and include that ADK version in its signed compatibility manifest and DevAI evaluation record.

[Sam123ben]

Non-production preprocessing candidate #18 is ready for review. It adds bounded skew estimation/correction and proves original-page coordinate inversion without selecting model weights or an inference runtime. All CI checks, including real PostgreSQL coverage and secret scanning, are green. The approval checklist and model/license/Temporal gates remain open.

[Sam123ben]

Temporal Rust qualification candidate PR #26 is ready for review as a draft. It pins official 0.8.0 crates, isolates the preview adapter, prevents cross-product/tenant workflow-ID collisions, bounds content-free workflow input and records the 300-page rollover/activity policy plus upgrade-soak and stable Go fallback.\n\nThis is evidence only, not production selection. Temporal/platform, security and SRE approval remain open, as do live replay, crash recovery, worker-versioning and 24-hour soak results.

[Sam123ben]

Proposed non-production slice for review: enforce the missing signed model-profile boundary in ocr-engine without selecting or loading any weight. The contract will accept only a bounded detached Ed25519 envelope over the exact manifest bytes, resolve a pinned key ID from a caller-supplied keyring, reject unknown/revoked keys and malformed/oversized manifests, and validate model/artifact SHA-256 identity, stage, calibration/dataset/license provenance, execution profile, rollback predecessor, and bounded named tensor contracts. It will not contain a download URL, model path supplied by a tenant, runtime loader, ONNX dependency, model artifact, production configuration, or deployment. An ADR and adversarial contract tests will accompany it. This gives #9 a concrete reviewable implementation of the manifest/tensor gate while model-license selection, benchmark thresholds, and runtime qualification remain explicitly open.

[Sam123ben]

The proposed signed-profile contract is implemented for review in PR #39: #39. It is deliberately stacked on green PR #38 so the lifecycle worker remains a separately reviewable dependency. The PR contains only the bounded verifier, immutable metadata API, ADR-0011, and adversarial tests; it does not select model weights or authorize a runtime/deployment.

[Sam123ben]

PR #39 is green on commit f206c3d, including Rust quality, fresh PostgreSQL coverage, committed-content secret scanning, and both container build/smoke/scan checks: https://github.com/tesserix/document-intelligence/actions/runs/33847779586. It remains proposed and review-gated; no model artifact, profile key, inference runtime, worker deployment, or production approval is implied by this evidence.

[Sam123ben]

Implementation status (2026-09-04): #38 (tenant-safe work dispatcher) and #40 (signed model-profile contract) are merged to main, each with green Rust quality, database coverage, container, and secret-scan gates. The service can now lease tenant-safe content-free work scopes and verify a bounded signed model profile before any artifact would be loaded.\n\nNo recognizer, model artifact, downloader, ONNX execution provider, worker deployment, or traffic rollout was added. The remaining launch gates are unchanged: legally approved model weights/license, trusted signing-key governance, benchmark/golden-dataset evidence, execution-provider qualification, and 24-hour Temporal replay/cancellation/heartbeat/crash-recovery qualification. This preserves the explicit no-fake-recognizer rule.

[Sam123ben]

Proposed safe hardening slice: the signed profile currently caps rank and each tensor dimension but does not cap the multiplicative tensor element envelope. I will add deterministic per-tensor and aggregate maximum-element validation using checked arithmetic, including dynamic maxima, so a valid signature cannot authorize infeasible runtime allocations. This affects only contract verification and adversarial tests; it will not add model weights, ONNX, a loader, a deployment, or any production selection.

[Sam123ben]

Merged hardening follow-up: #41 is now in main as 564f053. It bounds verified signed tensor contracts to 64 MiB each and 128 MiB across a profile, using checked arithmetic before any runtime allocation. All PR checks passed, including both container builds after refreshing the verified immutable Tesserix Debian runtime pin.

This changes no model/weights, ONNX loader, recognizer worker, or deployment. The native OCR deployment remains correctly blocked on the approved model, qualification, and operational gates listed in this issue.