[Feature]: LeRobot prepared-manifest receipts are written but nothing verifies a delivered import

[VARUN3WARE]

Problem

After import_lerobot_dataset finishes, a recipient of the delivery (landing MCAPs + prepared-manifest.json) cannot ask HFlow whether that delivery is still intact.

Schema-3 manifests already carry per-episode receipts (uri, content_id, size_bytes). The receipt docstring and docs/how-to/import-lerobot-v3.md say a delivery can be checked against the manifest without re-running import. There is no public library or CLI reader that performs that check:

hasattr(hflow, "verify_prepared_manifest")  -> False

Truncation, same-path byte replacement, or a missing landing object stay silent until some later consumer fails for unrelated reasons. Hand checks exist only inside tests (test_manifest_content_id_detects_a_truncated_episode).

This is the LeRobot twin of the snapshot receipt/verify split (#397/#401 writer, #428 reader). Writer landed in #389; verify was deferred there. Verified against current main (c6d6d3729d7d7107c72fe7de4a7caf11ee549167).

Proposed behavior

Add a first-party verification path that reads an existing schema-3 prepared-manifest.json and checks every listed episode against the landing files — without re-running conversion or touching the Hugging Face cache.

Smallest useful outcome: given a data root that already contains prepared-manifest.json and landing MCAPs, the caller learns pass/fail (and which episode failed) from the receipts HFlow already wrote (uri, content_id, size_bytes).

Suggested API (names flexible):

from hflow import verify_prepared_manifest

report = verify_prepared_manifest("file:///path/to/data-root")
# report.ok is True when every episode matches
# on failure: which uri, and whether missing / size mismatch / content_id mismatch

Suggested CLI (pick one shape):

hflow verify lerobot-import file:///path/to/data-root

Behavior contract:

Delivery state	Result
Unchanged after publish	Pass
Landing file missing	Fail, name the episode uri
Truncated / wrong size_bytes	Fail
Same-length byte replacement (content_id disagrees)	Fail
Extra files under landing/ not listed in the manifest	Ignore (or warn — confirm with maintainers)

Default verification must work offline against local roots and must not re-encode video or rewrite the manifest. Bucket roots should use the existing StorageRoot read path (same as import), without publishing new episodes.

Update docs/how-to/import-lerobot-v3.md so “can be checked against the manifest” points at this API/CLI.

Out of scope for the same change: snapshot verify (#428), resume CRC (#426), schema bumps, and adding source_episode_index unless maintainers want that as a tiny additive follow-up.

Alternatives considered

• Re-run import / re-convert — expensive; defeats the point of published receipts; needs source cache.
• Hand-check in tests only — already done (test_manifest_content_id_detects_a_truncated_episode); not a product surface for recipients of a bucket prefix or copied directory.
• Rely on snapshot verify (Snapshot integrity receipts are written but nothing reads them: no way to verify a delivered snapshot #428) — different artifact (format.json / snapshot tables vs LeRobot landing + prepared-manifest); does not cover import deliveries that never become a snapshot.
• Bump prepared-manifest schema first — unnecessary for the happy path; schema 3 already has the fields a reader needs (feat(import): per-episode receipts in prepared-manifest.json #389).

Compatibility and data-format impact

• No change to MCAP layout, episode bytes, catalog schema, or Airflow bundles.
• No prepared-manifest schema bump required unless maintainers later want additive fields (e.g. source_episode_index).
• Reader-only: consumes existing uri / content_id / size_bytes receipts.
• Unlisted extra files under landing/ should be ignored or warned (policy to confirm); verification must not rewrite the manifest or landing objects.
• Bucket roots: verification may read remote objects (same as other StorageRoot consumers); should not mutate the catalog or publish new episodes.

Contribution

Yes

[kstonekuan]

Accepted. Both quotes check out verbatim, which is what makes this more than a missing feature: docs/how-to/import-lerobot-v3.md:62 says a delivery "can be checked against the manifest without re-running the import", and the _PublishedEpisode docstring at src/hflow/importers/lerobot.py:96-103 says the three fields "carry everything a delivery-verification reader of the manifest needs". We documented a capability and shipped only the half that writes.

Pinning the current main SHA in the issue is the right habit. Line numbers here moved twice yesterday.

One thing to settle before either lands. This and #428 are the same shape over two different receipt formats, and #431 is the third member of the family. If they each invent their own CLI noun and their own report object, we end up with three verifiers that answer the same question in three vocabularies, and that is worse than having none.

So: whichever lands first sets the shape, and the second follows it. Concretely, agree across both on the CLI shape (hflow verify <what> <where>), what the report object looks like, what the exit code is for a delivery that fails verification as opposed to input we cannot read, and whether an unlisted extra file is a failure, a warning, or ignored. Your table already takes a position on that last one and flags it as needing confirmation, which is the right instinct: ignore it. A delivery is verified against what the manifest claims, and a file nobody listed is not evidence of anything.

@Sagar-024 has #428, so the two of you should agree on that shape rather than discovering the disagreement in review.

Do you want this one? Say so and I will assign it, which is the only thing that reserves an issue here.

[VARUN3WARE]

Yes — please assign this one to me.

Agreed on the shared shape before either this or #428 lands:

• CLI: hflow verify <what> <where> (same noun pattern for prepared-manifest and snapshot; Ingest transcodes a payload-damaged source without noticing, so the canonical episode gets a valid receipt for corrupt bytes #431 follows when it lands)
• Report object: one common shape both verifiers return (pass/fail, per-item findings, reason codes for missing / size / content_id), not two parallel types
• Exit codes: distinct failure for “delivery does not match receipts” vs “input we cannot read” (bad/missing manifest, unreadable root)
• Unlisted extras: ignore — verify against what the manifest claims; a file nobody listed is not evidence either way

I’ll sync with @Sagar-024 on that contract before opening a PR so we don’t invent two vocabularies in review.

[kstonekuan]

Assigned. That contract is right on all four points, and settling it in writing before either PR opens is the whole reason to have asked. Ignoring unlisted extras is the call I would have made too: verification answers whether the manifest's claims still hold, and a file nobody claimed is outside that question.

[VARUN3WARE]

@Sagar-024 You’ve got #428 and I’ve got this one, Kevin wants the shared verify shape locked in writing before either PR opens.

Proposed contract (from Kevin’s call + this thread):

1. CLI: hflow verify <what> <where>
2. One common report object both verifiers return (ok / per-item findings / reason codes for missing, size, content mismatch)
3. Exit codes: delivery fails verification ≠ input we cannot read
4. Unlisted extras: ignore

If that works for snapshot verify too, reply here (or on #428) and we can treat it as settled. If you want different names for <what> or the report fields, say what you prefer and I’ll match it, happy for either of us to land first as long as the second copies the same surface.

[Sagar-024]

@VARUN3WARE contract accepted on all four points. My side sets the shape. CLI is hflow verify snapshot here, yours is hflow verify lerobot-import. Report is a single VerificationReport with .ok and a .findings list where each item has .uri, .reason, and .detail. I will put this type in a shared file like src/hflow/verification.py so whoever lands second just imports it instead of copying.

Exit codes are 0 for clean, 1 for verified but damaged, and 2 for input we cannot read. Unlisted extras are ignored. One small addition: if there is no receipt at all, I lean exit code 3 for unverifiable so CI can tell it apart from broken. Let me know if you veto that. Land yours first if it is ready, I will match the surface either way.

[VARUN3WARE]

@Sagar-024 No veto on exit code 3, unverifiable (no receipt) should stay distinct from damaged and unreadable.

Accepted as the shared surface:

• CLI: hflow verify lerobot-import here / hflow verify snapshot on Snapshot integrity receipts are written but nothing reads them: no way to verify a delivered snapshot #428
• VerificationReport with .ok and .findings (.uri, .reason, .detail) in src/hflow/verification.py
• Exit: 0 clean, 1 damaged, 2 unreadable, 3 unverifiable
• Unlisted extras: ignore

I’ll land this side first against that contract and put the shared type in verification.py so #428 can import it.

[Sagar-024]

@VARUN3WARE to keep this from sitting on anyone, here is my default. My #428 build is audit-green and ready, and our contract says either of us landing first is fine. If you want to land #432 first, say so in the next few hours and I will hold and import your types. If I don't hear back I will push #428 now with verification.py written exactly to our contract, and you import Finding and VerificationReport from it on your side. If any field name feels wrong on your side I will change mine rather than make you fork the type. And when your PR is up ping me, I will review it so Kingston carries one less review this week.

[VARUN3WARE]

@Sagar-024 I'll land #432 first, pushing the PR now. Please hold #428 and import VerificationFinding / VerificationReport (plus VerificationStatus for exit 3) from src/hflow/verification.py. Field names match our contract; if anything is awkward for snapshot I'll adjust. Will ping you with the PR link for review.

[kstonekuan]

You two settled this without me, which is the right outcome, and the contract you landed on is the one I would have picked. One piece of review state so neither of you is waiting on the wrong thing.

#454 has changes requested. The contract is implemented as agreed, but it resolves each episode by the receipt's recorded absolute uri, so verifying a delivery that has moved reads the original rather than the copy in front of it. I damaged a copy and it still reported ok. Details on the PR.

@Sagar-024 so do not hold #428 for this to land. The shared types are not the part that needs rework, and the fix is inside _findings_for_episode_receipt. If you want to land first, land first; whoever is second imports the types rather than redefining them, and the order stops mattering.

Worth carrying into #428 either way: verify what is under the root you were handed, never what a receipt says about where the data used to live. A snapshot's integrity.tables entries are relative paths, so you may be safe by construction, but a test that copies the snapshot to a second root and verifies the copy is cheap and would prove it.

[kstonekuan]

Resolved by #454, which merged after rebasing onto #457's shared module. hflow verify lerobot-import is in, with the report shape and exit codes you and @Sagar-024 agreed here before either of you wrote code.

The verify family now covers both receipt formats: this and verify snapshot from #428, same VerificationReport, same reason strings, same 0/1/3 mapping.