Skip to content

fix(import): report the collector's success label instead of inventing one - #403

Merged
kstonekuan merged 2 commits into
Hebbian-Robotics:mainfrom
Sagar-024:feat/395-success-label
Sep 4, 2026
Merged

fix(import): report the collector's success label instead of inventing one#403
kstonekuan merged 2 commits into
Hebbian-Robotics:mainfrom
Sagar-024:feat/395-success-label

Conversation

@Sagar-024

Copy link
Copy Markdown
Contributor

Closes #395.

The importer hardcoded success: "true" into every episode/v1 record while its episodes query read seven columns and never touched an outcome field. FORMAT.md defines success as the collector-labeled outcome and says the record is copied/merged from the source; the v3 format carries that label as a next.success feature with per-episode stats/next.success/min|max aggregates, and lerobot/pusht declares it for all 206 episodes, uniformly False. So the shipped record contradicted the source it read, 100 percent of the time, on the ecosystem's reference dataset.

The fix, per the issue's open call: the importer reads stats/next.success/max as an eighth column when the source declares it, derives the episode outcome as MAX over the collector's frames (any success frame makes the episode a success), and stamps success together with success_derivation: "max(stats/next.success)" so the methodology travels with the data. When the source declares no outcome feature, the key is omitted entirely: FORMAT.md makes every episode/v1 key optional, and omission is the only encoding that cannot be mistaken for a collector's judgment. Column discovery runs first, because naming a missing column would fail the read, and a corpus without the feature is normal, not malformed.

The catalog promotion already renders an omitted key as SQL NULL; that behavior is pinned now, not changed. The transform's copy path is untouched.

CONVERTER_VERSION moves to v7 in the same commit: the label changes episode/v1 bytes that content_episode_id hashes.

The #395 fixture chain lands as tests through the real import, the real Episode reader, real Catalog.append_episode, and real SQL: a False,True transition reports true by MAX, an all-false source reports false, an unlabeled source omits the key and catalogs as NULL.

Gate: ruff, format, ty clean; 1530 passed / 6 skipped.

Refs #395. Third brick of the trust thread with #376 and #389.

Sagar-024 and others added 2 commits September 4, 2026 21:42
…g one

The importer hardcoded success=true into every episode/v1 record while
its episodes query read seven columns and never touched an outcome
field. FORMAT.md defines success as the collector-labeled outcome and
states the record is copied/merged from the source; the v3 format
carries that label as a next.success feature with per-episode
stats/next.success/min|max aggregates (lerobot/pusht declares it for
all 206 episodes, uniformly False).

The importer now reads stats/next.success/max as an eighth column when
the source declares it (column discovery first: a corpus without the
outcome feature is normal, not malformed), derives the episode outcome
as MAX over the episode's frames, and stamps success together with a
success_derivation naming the derivation so the methodology travels
with the data. When the source declares no outcome feature the key is
omitted entirely: FORMAT.md makes every episode/v1 key optional, and
omission is the only encoding that cannot be read as a collector's
judgment. The catalog promotion already renders an omitted key as SQL
NULL; that behavior is pinned, not changed.

CONVERTER_VERSION moves to v7 in the same commit: the label changes
episode/v1 bytes that content_episode_id hashes.

The fixture chain from Hebbian-Robotics#395 lands as tests through the real import,
Episode reader, Catalog.append_episode, and SQL: max over a False,True
transition reports true, an all-false source reports false, an
unlabeled source omits the key and catalogs as NULL.

Refs Hebbian-Robotics#395
…ty aggregate

The PR carried 34 files under .zcode/, including generated MCAPs, MP4s and
catalog parquets. Removed, and .zcode/ now sits beside .claude/ and .agents/
in .gitignore so it cannot recur for anyone using that tool.

An outcome column present but empty for an episode stamped success: "false",
since any([]) is False. That is the same invention the hardcoded "true" was,
one value over, and nothing caught it; it now has its own case.

Also lifts the column name and derivation string into constants, replacing a
chr(34) workaround for f-string quote nesting on 3.11.

@kstonekuan kstonekuan left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Merging with fixups in a6bc98b. The change is right and follows the direction closely: max when the source declares the feature, omission when it does not, the derivation stamped so the methodology travels, and v7 in the same commit.

The PR carried 34 files under .zcode/, including generated MCAPs, MP4s, and catalog parquets. Removed. .zcode/ now sits beside .claude/ and .agents/ in .gitignore, so it cannot recur for you or anyone else using that tool. That is the durable version of what your .gitignore edit on #389 was reaching for; worth adding it to your global gitignore too, since this repo is not the only one that would collect it. The checklist line about not adding generated media is the one to watch: git status before git add -A catches this in a second.

One real gap, closed in the fixup. An outcome column that exists but is empty for an episode stamped success: "false", because any([]) is False. That is the same invention the hardcoded "true" was, one value over: a declared feature with nothing recorded is not a judgment. Nothing caught it, so it has its own case now.

empty aggregate treated as a label   1 failed  <- was: 74 passed, nothing noticed

Everything else is properly held, and the is not None test rather than a truthy one is the detail that makes the False case work at all:

success hardcoded true again              2 failures
key stamped when the source has no label  1 failure
absent collapsed with False               1 failure
success_derivation not recorded           2 failures
outcome column never discovered           2 failures
CONVERTER_VERSION left at v6              1 failure

That last one is new and welcome. Three PRs in a row have needed a version bump and this is the first to pin it, so an omitted bump now fails rather than being caught in review.

Two derivation mutations go unnoticed, and that is correct rather than a gap. Flipping any to all, or to the last frame, changes nothing:

derivation flipped to MIN          75 passed, nothing noticed
derivation flipped to last frame   75 passed, nothing noticed

stats/next.success/max is already the per-episode maximum, so it arrives as a one-element list, exactly as your issue showed with ['False']. The any() reduces a single value. That means the reduction is defensive against a shape that does not occur, and the honest reading of success_derivation is "the source's own max", not a max we computed. Worth a sentence in the code saying so, since a future reader will otherwise assume the reduction is load-bearing and try to test it. I left the code alone; it is right, just easy to misread.

Small cleanup in the same fixup: the column name and derivation string are constants now, which removed the chr(34) workaround for f-string quote nesting on 3.11.

Gate on the merged result: ruff check, ruff format --check, ty check clean, 1531 passed / 6 skipped.

Closes #395. The trust thread reads well as a set now: #376 removed a stamp asserting an unmeasured value, #389 gave the delivery a checkable receipt, and this one stops the record speaking for the collector. The distinction you drew, that this one is about the data rather than the pipeline, is the right note to end on.

@kstonekuan
kstonekuan merged commit 44e4f50 into Hebbian-Robotics:main Sep 4, 2026
6 checks passed
@Sagar-024

Copy link
Copy Markdown
Contributor Author

Both fixups fair.
Thanks for closing the gap.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The LeRobot importer ignores the source's success label and hardcodes "true" for every episode

2 participants