Harden hackathon Rerun, Viewer, and SmolVLA workflow - #8
Merged
Conversation
Document the verified local SmolVLA training setup (Python 3.11 venv, vendored lerobot, the mandatory --dataset.video_backend=pyav workaround) and git-ignore outputs/ so training artifacts aren't committed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Fine-tune the pretrained lerobot/smolvla_base (--policy.path) rather than --policy.type=smolvla, which trains from scratch and won't learn a usable policy on ~150 episodes. Matches the official "warm-start from SO-101 base" recipe. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
smolvla_base expects cameras camera1/2/3; our data is front/side. Add the required --rename_map and --policy.empty_cameras=1 to the train + smoke-test commands. Full warm-start recipe verified end-to-end on MPS. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…/hackathon-readiness
There was a problem hiding this comment.
Pull request overview
This PR hardens the reBot Rerun/Query API workflow and adds a consistent operator-facing Viewer layout, while documenting a verified local SmolVLA fine-tuning recipe and clarifying hackathon demo evidence boundaries.
Changes:
- Add a purpose-built Rerun Viewer Blueprint and activate it during interactive recordings (while keeping
--no-viewerfully headless). - Log the vendored reBot URDF as actual meshes/transforms when possible, with a safe text fallback.
- Document SmolVLA warm-start training steps (MPS + PyAV) and add supporting docs/demo materials; update dependencies and ignore runtime artifacts.
Reviewed changes
Copilot reviewed 14 out of 16 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| SMOLVLA_TRAINING.md | New step-by-step local SmolVLA fine-tuning guide (warm-start, PyAV, camera rename/padding, MPS). |
| requirements.txt | Adds rerun-loader-urdf dependency for URDF mesh logging. |
| rebot_operator_kit/teleop_gui/training_workspace.py | Improves macOS Rerun native binary discovery (CLI vs app bundle). |
| rebot_operator_kit/teleop_gui/controlled_record.py | Improves macOS Rerun native binary discovery for verification and viewer usage. |
| p5_rerun_port/viewer_blueprint.py | New central Viewer Blueprint for cameras, URDF, metrics, and timeline. |
| p5_rerun_port/urdf_log.py | Logs URDF meshes/transforms via rerun-loader-urdf with fallback to text path. |
| p5_rerun_port/tests/test_rerun_hardening.py | New tests covering vendored URDF resolution/logging, Query API smoke, and blueprint/headless behavior. |
| p5_rerun_port/takes.py | Adds optional Viewer Blueprint initialization and activation to recording setup. |
| p5_rerun_port/README.md | Adds link to hackathon demo doc and notes Blueprint-enabled interactive recordings. |
| p5_rerun_port/constants.py | Prefers repo-vendored URDF path as first resolution candidate. |
| docs/TEAM.md | Updates SmolVLA guidance to point to the verified warm-start recipe. |
| docs/Rerun_bounty_progress.md | Marks Viewer Blueprint slice as done (synthetic) and updates bounty status notes. |
| docs/p5_rerun_port/README.md | Documents Viewer Blueprint workflow and confirms headless --no-viewer behavior. |
| docs/p5_rerun_port/examples/hackathon_smoke_query_report.md | New synthetic Query API report example with explicit evidence boundary. |
| docs/HACKATHON_SUBMISSION_DEMO.md | New hackathon pitch/demo script with explicit synthetic vs live claims and checklist. |
| .gitignore | Ignores .venv-lerobot/ and outputs/ artifacts. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| except Exception: | ||
| pass | ||
| rec.set_sinks(*sinks) | ||
| rec.set_sinks(*sinks, default_blueprint=blueprint) |
| @@ -40,6 +40,14 @@ | |||
|
|
|||
| # Prefer SDK URDF; fall back to env override. | |||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this integrates
mainVerification
17 passed: Rerun hardening + Viewer/library tests151 passed: Person 4 harness/episode/evaluation tests36 passed: rollout contract and safety testsPhysical boundary
No reBot serial hardware is connected to this Mac. Physical motion, live can sorting, physical replay, and learned-policy success are not claimed. Those remain gated by reviewed calibration, live hardware/camera checks, a clear workspace, and a dedicated e-stop operator.