AutoData

Turn scattered repair data into a vehicle-specific procedure, quote, and evidence-backed answer.

Finding the right repair answer is fragmented across manuals, generic search, diagnostic data, parts catalogs, and wiring diagrams. AutoData brings those sources together for one vehicle, preserves where the answer came from, and turns a plain-language request into a usable repair workflow.

The result is a vehicle-specific workspace with the relevant procedure, labor-and-parts quote, supporting source evidence, and clear review status. A fast lane makes the first useful answer available quickly; a deep lane keeps enriching it in the background.

Architecture · Local quick start · Wiki · Project #8

At a glance

• Vehicle-scoped: identity, configuration, procedures, specifications, diagnostics, evidence, and feedback stay attached to the requested vehicle projection.
• Fast to useful: the fast lane normalizes core facts and publishes an immutable, viewable revision without waiting for every enrichment section.
• Built to deepen: the deep lane adds documents, images, diagrams, search, embeddings, and quality review independently of the first viewable result.
• Safe by default: provenance, evidence, review status, entitlements, and source-rights boundaries are part of the platform contract.

The current verified local path is deterministic and uses PostgreSQL with pgvector, NATS JetStream, MinIO, fake source data, and a fake payment provider. It does not require cloud credentials. Generated procedures and source-derived results remain explicitly UNREVIEWED until an authorized human review changes their status.

The repository is a modular monorepo:

• apps/api-go — authenticated API, RBAC, entitlement-aware reads, feedback, and reviewer commands.
• workers/ingestion-python — provider-neutral source intake, normalization, provenance, fast-lane publication, and payment reconciliation.
• workers/enrichment-python — deep-lane section enrichment, evidence-backed search and quality processors, embeddings, immutable revisions, and publication outbox handling.
• packages/contracts — versioned API, event, and payment contracts.
• db/migrations — forward-only PostgreSQL and pgvector schema migrations.
• infra/compose and infra/k8s — local and Kubernetes-compatible topology.
• docs/architecture — canonical system, lifecycle, contract, and operations documentation.
• docs/github — repository and Project operating model.
• docs/agents/pre-implementation-gate.md — mandatory, agent-agnostic planning and GitHub/repository synchronization gate for implementation work.

For a guided introduction, start with the GitHub Wiki. For the authoritative technical details, use the linked documents under docs/.

Independent source connectors

AutoData calls the independent Bankone and Banktwo services through the shared provider-neutral Source Connector v1 contract. The configured defaults are https://bankone.cars.tk and https://banktwo.cars.tk, set independently by BANKONE_BASE_URL and BANKTWO_BASE_URL. AutoData owns source snapshots, normalization, provenance, and persisted jobs; each bank owns its upstream access and connector API. Wire slugs bankone and banktwo map to historical stored IDs autoapi and autoapitwo. DNS ownership, Vercel attachment, TLS, live authentication, deployments, and canaries remain pending. See the connector architecture for component locations and current status.

Current development slice

Provider-neutral catalog and Workshop

AutoData's active product path is the durable catalog API and Workshop. External applications select year → make → model → configuration → article through versioned /v1/catalog/* routes. Workshop at /workshop/ is the operator UI for the same cascade. The chatbot is not part of the active product path (Issue #113).

Source material comes from independent Bankone and Banktwo connectors through Source Connector v1. Configure BANKONE_BASE_URL and BANKTWO_BASE_URL independently. Live DNS, deployments, and service access remain unverified.

Normalized articles keep source order, store images locally behind opaque tokens, and retain immutable original provider content for review. Multi- component composition stays response-only and is not persisted as a derived catalog article.

Canonical records:

• provider-neutral catalog API
• independent bank connectors
• chatbot removal plan

Start the local stack

The deterministic local path uses PostgreSQL with pgvector, NATS JetStream, MinIO, fake source data, and a fake payment provider. It does not require cloud credentials:

AUTODATA_POSTGRES_PASSWORD=local-dev-only \
AUTODATA_MINIO_ROOT_USER=localadmin \
AUTODATA_MINIO_ROOT_PASSWORD=local-dev-password \
docker compose -f infra/compose/compose.yaml up -d \
  postgres nats minio migration-runner

AUTODATA_POSTGRES_PASSWORD=local-dev-only \
AUTODATA_MINIO_ROOT_USER=localadmin \
AUTODATA_MINIO_ROOT_PASSWORD=local-dev-password \
docker compose -f infra/compose/compose.yaml run --rm ingestion-smoke

The smoke command verifies the deterministic purchase, entitlement, viewable revision, PostgreSQL records, MinIO source object, and dataset.viewable JetStream event. The full developer workflows and recovery checks are in docs/architecture/infrastructure-and-dev.md.

The API serves its catalog-media contract at /openapi.json and /openapi.yaml, with same-origin Swagger UI at /swagger/. Configure a stable AUTODATA_IMAGE_URL_KEY through the local environment or secret manager (shared by all API replicas); without a stable key, image URLs intentionally fail closed and are omitted from article responses. Provider images are fetched only during ingestion and displayed from AutoData's local object store.

Open the local AutoData Workshop after the API container is running. External applications use the provider-neutral /v1/catalog/years cascade to select a vehicle, browse its article index, and request normalized individual article details. /dashboard/ is a compact landing page linking to Workshop, Swagger, and the catalog API.

Configure BANKONE_BASE_URL and BANKTWO_BASE_URL independently; these intended domain defaults do not prove live DNS or service availability. Runtime source access uses the shared Source Connector v1 contract and does not require a bank service to be built into the AutoData stack.

Validate a heterogeneous source drop

The source normalizer accepts mixed JSON/API envelopes, XML, CSV, HTML, plain text, PDF, SVG, and unsupported media without relying on filenames. It retains raw bytes and provenance, emits typed candidates where the shape is recognized, extracts literal HTML/plain text, page-level text from text-based PDFs, and literal SVG labels into reviewable evidence. Configured raster images can use the worker's Tesseract OCR boundary to emit confidence-aware region evidence. Mixed or scanned PDFs rasterize only pages without native text and send those pages through the same OCR/evidence boundary. JSON envelopes can also carry explicit embedded html or base64 pdf resources; these are adapted through the same document/PDF path with locators such as body.html:{document_id} and body.pdf:{document_id}:page:{n} while the outer JSON remains the source content address. Ambiguous or unknown material is routed to review. New source media types can register an adapter without changing the connector or event contract; if an optional embedded-resource, PDF rasterizer, or OCR capability is unavailable, the original outer payload remains content-addressed with review metadata.

For a local-only source directory:

PYTHONPATH=workers/ingestion-python/src \
python3 scripts/dev/normalize_source_directory.py "sample data" --region US

The sample data/ directory is intentionally not part of the public repository. Do not stage or publish source payloads unless their redistribution terms have been explicitly approved. The Compose-mounted persistence form and its review semantics are documented in the infrastructure guide.

The normalizer also emits a source_report array with one payload-free record per resource. It includes the source URI/version, media type, content hash, artifact kind, candidate counts and kinds, extraction status, and stable review reasons. When persistence is enabled, conflicts and quarantine reasons are also written as idempotent source_review_items records linked to source snapshots and extraction evidence. Similar-article quarantine entries are coalesced with their article-similarity conflict so the review queue does not duplicate one ambiguity. Embedded-resource summaries include their locator and content hash but never copy raw source contents or credentials.

The same pipeline can run through the ingestion worker boundary instead of the developer script:

AUTODATA_POSTGRES_PASSWORD=local-dev-only \
AUTODATA_MINIO_ROOT_USER=localadmin \
AUTODATA_MINIO_ROOT_PASSWORD=local-dev-password \
docker compose -f infra/compose/compose.yaml run --rm --no-deps \
  -v "$PWD/sample data:/sample-data:ro" \
  -e AUTODATA_SOURCE_DIRECTORY=/sample-data \
  -e AUTODATA_SOURCE_VERSION=local-sample-v1 \
  -e AUTODATA_SOURCE_REGION=US \
  -e AUTODATA_SOURCE_PERSIST=0 \
  -e AUTODATA_WORKER_ONCE=1 \
  ingestion-worker python -m autodata_ingestion.worker

Set AUTODATA_SOURCE_PERSIST=1 only when the local PostgreSQL and MinIO connection variables are also provided. The worker reports bundle readiness and quality/review status separately.

For one HTTP(S) source resource, set AUTODATA_SOURCE_URI instead of AUTODATA_SOURCE_DIRECTORY. The connector captures the response status, non-secret response headers, redirect target, and raw bytes before the same content-first classification and normalization path. Source authentication, if required, belongs in secret-managed AUTODATA_SOURCE_REQUEST_HEADERS_JSON; do not commit header values or credentials to the repository.

Normalize a vehicle list into stable selection JSON without starting the container stack:

AUTODATA_WORKER_ONCE=1 \
AUTODATA_VEHICLE_LIST_JSON='[{"model_year":"99","make":"Chevy","model":"Silverado 1500","region":"US","drivetrain":"2wd"},{"year":1999,"make":"Chevrolet","model":"Silverado 1500","region":"US","drivetrain":"2WD","engine_displacement_l":5.3}]' \
PYTHONPATH=workers/ingestion-python/src \
python3 -m autodata_ingestion.worker

Set AUTODATA_SOURCE_PERSIST=1 when the PostgreSQL and MinIO variables are available to persist the list as an immutable source snapshot. Each row is stored with evidence and an identity observation; richer rows add a configuration beneath the existing vehicle_id instead of creating another vehicle family. AUTODATA_VEHICLE_LIST_SOURCE_URI and AUTODATA_SOURCE_VERSION identify the list source for replay and audit.

For a complete local Bankone export, use the batch runner. It discovers a vehicle bundle from each directory containing name.json, derives selector configurations from the split name.json and motorvehicles.json responses, and processes every other file in that directory through the universal source adapter. A source root containing name.json is treated as one vehicle; a catalog root containing child bundles is processed one vehicle at a time. If multiple discovered bundles normalize to the same vehicle family, they are merged into one batch and all source directories are normalized together, so article deduplication and provenance cover the complete drop set. A selector export may also arrive first, without article bundles. Pass it with --selector-json; nested models and engines are flattened into configuration observations, and every selector vehicle is retained as pending_source until its article bundle arrives. A selector vehicle without a matching source bundle is never silently skipped:

PYTHONPATH=workers/ingestion-python/src \
python3 scripts/dev/ingest_bankone_batch.py "sample data" \
  --region US \
  --source-version bankone-local-v1

Selector-only catalog import (identity/configuration stage):

PYTHONPATH=workers/ingestion-python/src \
python3 scripts/dev/ingest_bankone_batch.py "bankone-export" \
  --selector-json "bankone-export/vehicles.json" \
  --region US \
  --source-version bankone-selector-v1

When the corresponding per-vehicle directories are added beneath the same source root, rerunning the command merges the selector observations with the bundle-derived trims and engines, then normalizes and persists each available article independently. Replays use the stable vehicle, source, article, and locator identities; near-duplicate articles remain linked to their canonical record for review rather than being published twice. The batch result is completed only when every planned vehicle was processed without a review or failure condition; pending_source, needs_review, and failed remain explicit per-vehicle and aggregate outcomes.

Add --persist only when PostgreSQL and MinIO are available through the local environment. This persists the derived selector rows and each vehicle's source snapshots, evidence, normalized articles, duplicate links, and review items. The command continues across vehicle bundles and reports per-vehicle failure or review status. A remote Bankone connector must provide the same immutable bundle shape; this local runner does not guess undocumented remote endpoint paths.

Each processed result includes article_coverage: raw candidate count, raw unique article IDs, normalized unique IDs, review/quarantine IDs, and an unaccounted_unique_ids list. A valid complete bundle has an empty unaccounted_unique_ids list; near-duplicate records are accounted for in the review set rather than silently dropped.

For one target article, set AUTODATA_ARTICLE_URI and provide the target vehicle as JSON. The worker returns the normalized article records together with their source evidence; source credentials, if required, remain in the secret-managed request-header variable:

AUTODATA_WORKER_ONCE=1 \
AUTODATA_ARTICLE_URI=https://example.test/article \
AUTODATA_ARTICLE_VEHICLE_JSON='{"year":1999,"make":"Chevrolet","model":"Silverado 1500","region":"US","drivetrain":"2WD"}' \
PYTHONPATH=workers/ingestion-python/src \
python3 -m autodata_ingestion.worker

The same worker boundary is available through the local Go API when the Compose stack is running. POST /article-intakes accepts {"source_uri":"https://...","vehicle":{...}} with an Idempotency-Key and returns the normalized article, vehicle association, and evidence JSON. It requires the ingestion_operator role. POST /knowledge-queries accepts the vehicle/query request shape below with an Idempotency-Key; it requires the dataset_viewer role and returns cache_hit when the indexed catalog matches, or fetched after one bounded source fallback and normal intake. The API service forwards these calls only to the internal ingestion-http service. Set AUTODATA_INGESTION_INTERNAL_TOKEN through the environment or a secret manager when the internal network is not otherwise trusted; never put that token, source headers, or provider keys in the repository.

For vehicle-scoped keyword lookup, provide a normalized catalog in the request. A catalog hit is returned immediately without a source request:

AUTODATA_WORKER_ONCE=1 \
AUTODATA_KNOWLEDGE_REQUEST_JSON='{"vehicle":{"year":1999,"make":"Chevy","model":"Silverado 1500","region":"US"},"query":"brake connector","catalog":[{"vehicle_key":"chevrolet-silverado-1500-1999-us","kind":"article","article":{"article_id":"TSB-42","title":"Brake connector bulletin"},"evidence":[]}]}' \
PYTHONPATH=workers/ingestion-python/src \
python3 -m autodata_ingestion.worker

On a catalog miss, set source_uri_template in the request or AUTODATA_KNOWLEDGE_SOURCE_URI_TEMPLATE in the environment. The HTTP source template may use {vehicle_key}, {year}, {make}, {model}, {region}, {body_style}, {trim}, {drivetrain}, {engine_displacement_l}, {query}, and {keywords}. The worker URL-escapes those values, fetches one bounded source resource, verifies the returned vehicle, and returns the normalized article with evidence. Source responses are never treated as a match unless the requested vehicle and query both pass the intake boundary.

When catalog is omitted from a knowledge request, the worker first performs an indexed PostgreSQL lookup by canonical vehicle_key and ranks the normalized records locally. Component job-plan queries add bounded title filters before applying the result limit, so a requested oil- or water-pump article is not missed merely because it sorts beyond the first page of a large catalog. Unfiltered reads exclude linked duplicates; all reads exclude records without persisted evidence and source snapshots marked for takedown. Only when the database lookup has no matching result does it resolve and fetch the configured source. Supplying catalog: [] deliberately bypasses the database lookup and is useful for controlled fallback tests.

Mercury-2 is optional and advisory. It can adjudicate ambiguous vehicle identity matches and, when explicitly enabled, extract typed candidates from otherwise-unrecognized structured source shapes. Deterministic normalization, provenance, evidence, and review gates remain authoritative; the model cannot publish directly to canonical tables. Enable it only through secret-managed environment variables; never place the API key in Compose files, source files, README examples, or Git history:

export INCEPTION_API_KEY='<set-locally-or-through-a-secret-manager>'
export INCEPTION_API_BASE_URL='<provider-endpoint>'
export AUTODATA_MERCURY2_EXTRACTION_ENABLED=1

The source extractor is called only for structured artifacts that have no deterministic typed candidates. It is bounded by AUTODATA_MERCURY2_EXTRACTION_MAX_INPUT_BYTES and AUTODATA_MERCURY2_EXTRACTION_MAX_CANDIDATES; a missing configuration, timeout, invalid response, or empty proposal leaves the raw source and marks the artifact needs_review for replay.

Run tests

(cd apps/api-go && go test ./...)
(cd packages/contracts/go && go test ./...)
PYTHONPATH=workers/ingestion-python/src python3 -m pytest workers/ingestion-python/tests -q
PYTHONPATH=workers/enrichment-python/src python3 -m pytest workers/enrichment-python/tests -q
PYTHONPATH=workers/ingestion-python/src python3 -m pytest scripts/dev/test_*.py -q
python3 scripts/contracts/test_contracts.py
python3 scripts/dev/test_k8s_manifests.py

Architecture and delivery decisions are maintained in the linked canonical documents rather than duplicated in this index.