Merged
Conversation
…sts (#1350) Busy-wait loops with no timeout could hang forever. Use threading.Event for efficient blocking with 1s timeouts that fail clearly on expiry.
* added twist base protocol spec * created mock twist base adapter * added twistbase ConnectedHardware type * updated coordinator to support twist messages * added flowbase adapter * added blueprint and test script for testing * mypy test fixes * fix validates the adapter/hardware-type pairing upfront with a clear error * fixed redundant blueprint assignments * fixed mypy type ignore flags * added thread locks to velocity read write for flowbase adapter * fix mypy errors: portal import-untyped and adapter property override * added echo cmd vel script back for testing will be deprecated * removed echo_cmd_vel test script
…ntegration (#1351) Move adding_a_custom_arm.md from docs/development/ to docs/capabilities/manipulation/ where it belongs. Remove the outdated depth_camera_integration.md and fix resulting broken links.
* Moving towards deprecating `Agent`. It has been efectivelly split between `McpClient` and `McpServer`. * McpServer exposes all the `@skill` functions as MCP tools. * McpClient starts a langchain agent which coordinates calling tools from McpServer. * McpClient is not strictly necessary. You can register the MCP server in claude code and make it call tools. * McpClient also listens to `/human_input` so it can be used through the `humancli` like `Agent` can.
* added go2 preflight checklist * typo --------- Co-authored-by: Paul Nechifor <paul@nechifor.net>
- Add docs/platforms/humanoid/g1/index.md with full G1 guide - Update README.md to link G1 to new docs instead of todo.md - Covers: installation, simulation, real robot, agentic control, arm gestures, movement modes, keyboard teleop, all blueprints Closes DIM-576
…1348) * feat(doclinks): validate and resolve .md links, fix broken doc links Add Pattern 3 to doclinks that validates existing .md links in docs: - Resolves relative .md paths to absolute links - Validates absolute .md links exist on disk - Falls back to doc_index search for broken links with disambiguation - Handles index.md files by searching parent dir name - Scores candidates by directory overlap + filename match Delete duplicate docs/usage/transports.md (identical to transports/index.md). Fixes 5 broken links across docs/ (agents/docs/index.md, capabilities/ navigation/readme.md, usage/lcm.md). * refactor
* add missing command * Apply suggestion from @greptile-apps[bot] Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> * fix * Update bin/gen-diagrams Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> --------- Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* feat(blueprints): add scratch * feat(blueprints): be able to run modules too * remove scratch
* Remove asyncio from dependencies * UV lock --------- Co-authored-by: s <pomichterstash@gmail.com> Co-authored-by: Sam Bull <Sam.B@snowfalltravel.com>
Add tested and supported hardware specifications: - GPU: NVIDIA RTX 4070+ recommended - CPU: Intel i7+ or AMD Ryzen 7+ - Memory: 16GB minimum, 32GB+ recommended - Storage: 50GB+ SSD recommended Includes note about older hardware (3000-series GPUs) not being guaranteed.
* time system checks * refactor(system_configurator): split into package with separate modules Split the monolithic system_configurator.py into a package for better organization as more configurators are added. base.py has the ABC and helpers, clock_sync.py has ClockSyncConfigurator, lcm.py has LCM configurators. __init__.py re-exports everything for backward compat. * refactor: remove unitree_clock_sync wrapper, use system_checks directly Drop requirements.py and its tests — the global_config-based connection type check was unnecessary. Blueprints now call system_checks(ClockSyncConfigurator()) directly. * refactor: consolidate human-readable formatters into dimos/utils/human.py Replace 6 scattered duration/byte formatting implementations with two shared functions: human_duration() and human_bytes(). * fix(clock_sync): use ntpdate/sntp instead of systemd for clock fix Replace hardcoded timedatectl + systemd-timesyncd with a fallback chain: ntpdate > sntp > date -s. Avoids touching systemd on systems where timesyncd isn't installed (e.g. Arch). Also removes duplicate warning printed from check() (explanation() already handles it). * refactor: add human_number() for SI-suffixed number formatting Move msgs/sec formatter from benchmark heatmap into dimos/utils/human.py as human_number() (42, 1.5k, 3.2M). * blueprint configurator support * clock configurator refactor * configurator API comment * small fixes * 200ms time tolerance * small fixes * use typer.confirm and logger instead of input/print in system configurator Addresses PR review feedback: replace manual input() with typer.confirm() and all print() calls with proper logging. * remove markdown paths-ignore from code-cleanup workflow The doclinks pre-commit hook needs to run on markdown changes too. * removed autoconf from cli * feat(blueprints): separate .configurators() from .requirements() Address PR review: add dedicated .configurators() chaining method for SystemConfigurator instances, keeping .requirements() for plain callable checks only. * configurators list cleanup * blueprints gen bugfix
* handle the non-tty case * fix tests * fixup tests
* Resolve annotations using namespaces from the full MRO chain * refactored manipulation module to make it generic * pick and place module added * updated blueprints * added unit test * Partial orientation specification will preserve current orientation for missing angles * CI code cleanup * fix import error * fix type checking for RPC client
* Feat: modified buttons to publish absolute trigger values * Fix: control loop start must be in start * feat: packing analog triggers iwith buttons in UInt32 * for xarm7 * Feat: added gripper as hardware_component * Feat: gripper trigger decode * Fix: pre-commit fixes * comments update * fix: bit conversion amth + comments * fix: comments and errors * fix comments + pre-commit errors * fix: blueprint + mypy tests * CI code cleanup * Fix: no gripper as component, adding with joints * feat: add literal for hand str * fix: arm and gripper joint params * CI code cleanup * Misc: docstrings
…ver (#1385) * initial commit * feat: chhange from deno to python server - using fastAPI * feat: sending encoded msgs directly - no topic * feat: adding https support * Feat: added fastapi replacing deno for quest * feat: sending encoded raw msgs * fis: pre-commit errors * misc: readme updates * misc: debug prints * fix: remove unused In * fix: path correction * Misc: path for ssl certs * fix: catch frame_id if its not left or right
- Remove uv pip install from python and dev Dockerfiles - Install deps at test time with uv sync --all-extras --no-extra dds --frozen - Remove pyproject.toml from docker rebuild triggers - Rename run-ros-tests to run-tests - Fixes stale deps when pyproject.toml changes without Dockerfile changes
* feat(msgs): add PointStamped geometry message type adds geometry_msgs.PointStamped following the PoseStamped pattern: - inherits from dimos_lcm PointStamped + Timestamped mixin - lcm_encode/lcm_decode for binary roundtrip - to_rerun() → rr.Points3D - to_pose_stamped() → PoseStamped with identity orientation - plum.dispatch constructors (x/y/z, Vector3, list, kwargs) - conforms to DimosMsg protocol - 30 unit tests enables the rerun viewer click-to-navigate pipeline: viewer publishes PointStamped to /clicked_point via LCM, any module subscribes via LCMTransport and converts to PoseStamped for navigation. DIM-643 * CI code cleanup * refactor(msgs): drop plum.dispatch, use Point(LCMPoint) base - replace multi-dispatch constructors with single __init__(x, y, z, ts, frame_id) - inherit from Point(LCMPoint) instead of Vector3(LCMVector3) - add Point wrapper class for geometry_msgs.Point - simplify tests to focus on LCM roundtrip per review feedback addresses review comments from leshy on PR #1388 * test(msgs): simplify PointStamped tests to match TwistStamped pattern 3 focused tests: lcm roundtrip, Point inheritance, PoseStamped conversion * style(msgs): ruff format and lint fixes * refactor(msgs): use Timestamped.ros_timestamp() instead of local sec_nsec * refactor(msgs): move Point to separate file Point is a separate LCM message type, should have its own file like Vector3, Quaternion, Pose, etc. * fix(msgs): resolve mypy errors in Point/PointStamped - add type: ignore[misc] for LCMPoint subclass (generated code has Any type) - add PoseStamped to TYPE_CHECKING import block - fix test import: Point from Point module, not PointStamped module --------- Co-authored-by: spomichter <12108168+spomichter@users.noreply.github.com>
add types filter to pull_request triggers so CI doesn't run on draft PRs. only triggers on: opened, synchronize, reopened, ready_for_review
* feat(core): add per-worker resource monitor with dtop TUI Adds psutil-based resource monitoring to ModuleCoordinator.loop(), collecting CPU, memory (PSS/USS/RSS/VMS), threads, children, FDs, and IO stats per worker process every 2s. Stats are published over LCM and viewable with the new `dtop` CLI tool. - WorkerStats dataclass and collect_stats() on Worker/WorkerManager - ResourceLogger protocol with LCM and structlog implementations - dtop: live Textual TUI subscribing to /dimos/resource_stats - psutil added as explicit dependency - Bump smart blueprint to 7 workers * fix(dtop): fix cpu_percent always-zero bug and improve TUI Cache psutil.Process objects across snapshots so cpu_percent(interval=None) has a previous sample to diff against. Fix wrong module name in docstring, remove dead _snap_count state, and extend color gradient to cyan→yellow→red. * faster interval * need to refactor the resource monitor * worker vs generic process data types * refactor, fixing voxel tests * expensive carving test marked as tool * better dtop * dtop visuals * visual fixes * drop wrap * add "dimos top" * workerstats moved to worker * circular import fix * Fix type narrowing for Process.pid instead of type: ignore * refactored resourcemonitor * small decorator rewrite * global config monitor stats * dtop shows waiting for stats * stats disabled by default * renamed stats to dtop * better drop init msg * fix ttl_cache tests to match new _TtlCacheWrapper API
* feat(testing): add coverage * wait for native file
feat(cli): dimos restart command (DIM-683)
The multicast configurator checked whether a 224.0.0.0/4 route existed but not which interface it was on. On macOS the route often exists on en0, causing cross-process LCM communication to fail silently.
* fix: symlink libpython for MuJoCo mjpython on macOS with uv When Python is installed via uv, mjpython fails because it expects libpython at .venv/lib/ but uv places it in its own managed directory. Add a LibPythonConfiguratorMacOS system configurator that auto-creates the symlink during system setup before launching the MuJoCo subprocess. * CI code cleanup * fix: handle broken symlinks and OSError in libpython configurator * CI code cleanup * fix: update test to expect LibPythonConfiguratorMacOS in Darwin checks * fix: detect broken symlinks in libpython check target.exists() follows symlinks and returns False for dangling ones, so the extra is_symlink() guard was silently skipping broken symlinks. The fix() method already handles unlinking stale symlinks before recreating them. --------- Co-authored-by: spomichter <pomichterstash@gmail.com>
* feat: log MCP tool calls and agent messages to per-run JSONL MCP server: log every tool call (name, args), response (truncated to 200 chars), duration, and errors. Previously only errors were logged. Agent: pretty_print_langchain_message now also writes to structlog so agent conversation (human, AI, tool calls, tool responses) appears in per-run JSONL logs alongside module-level events. Both are now visible via dimos log. * fix: clean up agent message JSONL logging - Filter base64 image data once at content extraction instead of at each usage site (console output and JSONL logging). - Simplify _log_message: replace if/elif/else branches with a single logger.info call using conditional kwargs dict.
* fix: rate-limit rerun bridge to prevent viewer OOM Add per-entity-path rate limiting (default 10 Hz) to _on_message. Without this, 30fps camera streams flood the viewer with data faster than its memory eviction can keep up, causing unbounded memory growth in the renderer's texture cache (not governed by memory_limit). Configurable via min_interval_sec on the bridge Config. Set to 0 to disable rate limiting for low-bandwidth deployments. * fix: init _last_log in start() instead of lazy hasattr check Move rate limiter dict initialization from per-message hasattr check to start(), before subscriptions begin. Removes per-call overhead and a potential race where two threads could overwrite each other's dict.
* feat(cli): dimos log command for viewing per-run logs (DIM-698) Add log_viewer.py with log resolution, JSONL formatting, and tail -f follow mode. CLI wrapper in dimos.py is a thin 15-line command. Usage: dimos log # last 50 lines, human-readable dimos log -f # follow in real time dimos log -n 100 # last N lines dimos log --all # full log dimos log --json # raw JSONL dimos log --run <id> # specific run Closes DIM-698 * refactor: cleaner log_viewer — deque tail, extracted helper, keyword-only arg * fix: keyword-only json_output in format_line calls * fix: dimos log -f exits cleanly on Ctrl+C Use SIGINT handler + stop callback instead of KeyboardInterrupt. The follow_log generator checks the stop flag every 0.1s and exits cleanly, ensuring the file handle is closed and the terminal is restored. * CI code cleanup --------- Co-authored-by: spomichter <12108168+spomichter@users.noreply.github.com>
* fix: resolve camera intrinsics * fix: resolve sim intrinsics via hasattr
* chore: add GitHub issue templates for bug reports and feature requests * chore: add title prefixes and remove auto-labels from issue templates * chore: reorder bug report template and make robot fields optional
* chore: add GitHub issue templates for bug reports and feature requests * chore: add title prefixes and remove auto-labels from issue templates * fix: G1 agent RPC wiring and system prompt - Add G1ConnectionBase ABC so skill container RPC calls resolve for both G1Connection (hardware) and G1SimConnection (simulation) - Add G1-specific system prompt describing the humanoid's actual skills (move, arm gestures, mode commands) * fix: register speak_skill and export G1ConnectionBase Add speak_skill() to G1 agentic skills so the agent can use the speak tool referenced in the system prompt. Export G1ConnectionBase in __all__ since it is the public contract for sim and skill wiring. * fix: add abstract start/stop to G1ConnectionBase The test_module_has_start_and_stop test requires all Module subclasses to declare start and stop methods. Add them as abstract methods on the base class to satisfy the test and enforce the contract on subclasses. * fix: resolve mypy safe-super errors in G1ConnectionBase Give abstract start()/stop() methods a non-trivial body that chains to Module via super(), fixing 4 safe-super mypy errors. * Revert to dev --------- Co-authored-by: stash <pomichterstash@gmail.com>
* fix: remove redundant autoconf call from LCMService.start() The blueprint system already runs configure_system via _run_configurators before deploying any modules to workers. The per-service call caused repeated prompts (once per LCMService instance across all worker processes). * fix: remove redundant autoconf from LCMService.start() The blueprint system already runs configure_system via _run_configurators before deploying any modules to workers. The per-service call caused repeated "Apply these changes now?" prompts (once per LCMService instance across all worker processes). Also removes the now-dead LCMConfig.autoconf field and cleans up all callsites that passed it. * CI code cleanup * docs: remove stale autoconf=True from code snippets * fix: add autoconf(check_only=True) to standalone CLI tools Standalone tools (rerun-bridge, dtop, lcmspy, topic echo) bypass the blueprint system and no longer get autoconf() from LCMService.start(). Add check_only=True calls so they print warnings about missing system config (multicast, sysctl buffers) without blocking on interactive prompts. --------- Co-authored-by: spomichter <pomichterstash@gmail.com>
* docs: add AGENTS.md for AI agent onboarding Covers repo structure, architecture basics, CLI usage (daemon mode, status, stop), testing, code style, git workflow, common pitfalls, viewer backends, and replay mode. Points to docs/ for deep dives. * docs: fix testing section — pytest defaults to fast tests via addopts * docs: use uv run for pytest/mypy in AGENTS.md * Draft agents.md * cleaned up / handwrote agents.md * agents changes
* refactor(perception): split temporal memory god class into clean components Architecture refactor of the temporal memory system: - TemporalMemory(Module): thin orchestrator with reactive RxPY pipeline - FrameWindowAccumulator: bounded frame buffering + windowing (pure logic) - WindowAnalyzer: isolated VLM interaction layer (testable, stateless) - TemporalState: typed dataclass with thread-safe snapshot support - EntityGraphDB: cleaned up, removed dead semantic_relations table Config changes: - TemporalMemoryConfig exposes ALL VLM frequency knobs at top level - Added db_dir, new_memory, stale_scene_threshold, max_distance_pairs Storage simplified to two outputs: - Per-run: temporal_memory.jsonl with raw VLM responses (via get_run_log_dir) - Persistent: memory/temporal/entity_graph.db (survives across runs) - Removed: state.json, entities.json, frames_index.jsonl, evidence.jsonl Other changes: - extract_time_window uses regex-only (removed wasteful VLM image call) - Added Rerun GraphNodes/GraphEdges entity graph visualization - Removed temporal_memory_deploy.py (use blueprints/autoconnect) - Removed temporal_memory_example.py (replaced by tests) - Removed temporal_utils/state.py (replaced by TemporalState) - 29 unit tests covering all components, all passing * docs(temporal-memory): rewrite README with config flags, architecture, storage docs - Replace stale README.md referencing old artifacts (evidence.jsonl, state.json, entities.json, frames_index.jsonl, output_dir) - Document all TemporalMemoryConfig flags with defaults and descriptions - Add architecture diagram showing 5-component split - Document storage outputs (per-run JSONL + persistent SQLite DB) - Add VLM call budget section for cost estimation - Add standalone blueprint composition example - Remove redundant temporal_memory.md (was copy of old README) * fix(temporal-memory): JSONL logging, Rerun viz, and query diagnostics Three bugs found during live testing on real hardware: 1. JSONL not written in worker processes: get_run_log_dir() only checks the in-process global, which is not set in forkserver workers. Fall back to DIMOS_RUN_LOG_DIR env var (inherited). 2. Rerun visualization invisible: rr.log() is a no-op without an active recording context. Worker processes don't inherit the bridge's Rerun connection. Add _ensure_rerun() that lazily connects to the gRPC server at 127.0.0.1:9876. 3. Query appears to hang with no feedback: add logging before graph context build and VLM call so users can see progress. * fix(temporal-memory): move persistent DB to XDG state dir, fix logging, fix Rerun - Persistent DB moved from DIMOS_PROJECT_ROOT/memory/temporal/ to ~/.local/state/dimos/temporal_memory/ (XDG compliant, same root as per-run logs). Repo root is not the right place for runtime state. - Log both paths at startup so users can always find their data. - Warn if no run log dir found (JSONL logging disabled). - Revert wrong Rerun connect: other DimOS modules just call rr.log() and let it no-op if no recording exists. Bridge is the canonical Rerun entry point. TemporalMemory follows the same pattern. - Remove unused DIMOS_PROJECT_ROOT import. - Update tests to remove stale DIMOS_PROJECT_ROOT patches. - Update README with new DB location. * feat(temporal-memory): 3D entity markers on Rerun map via odometry Replace the 2D GraphNodes/GraphEdges visualization (separate panel, disconnected from the scene) with 3D entity markers overlaid on the world map using the robot's odometry position. How it works: - TemporalMemory now subscribes to odometry: In[Odometry] - Tracks robot (x, y, z) position continuously - When entities are detected, their world position (robot's position at detection time) is stored in the entity DB metadata - After each window analysis, publishes EntityMarkers on an output stream — the Rerun bridge auto-picks it up via to_rerun() - Renders as rr.Points3D with per-entity labels and colors: person=red, object=green, location=blue New files: - dimos/msgs/visualization_msgs/EntityMarkers.py — labeled 3D markers with JSON-over-LCM serialization and to_rerun() → rr.Points3D - dimos/msgs/visualization_msgs/__init__.py Changes: - TemporalMemory: +odometry input, +entity_markers output, robot pose tracking, _publish_entity_markers() replaces _visualize_graph() - EntityGraphDB.save_window_data(): accepts metadata= kwarg, passes world position through to upsert_entity() for all entity types - Tests: new TestEntityMarkers with publish + to_rerun assertions * CI code cleanup * fix(entity-markers): add lcm_encode/decode so bridge can deserialize markers EntityMarkers was getting pLCMTransport (pickle) because it had no lcm_encode method. The bridge's LCM pubsub couldn't decode pickle messages. Added lcm_encode/decode aliases and exported the class from visualization_msgs/__init__.py so resolve_msg_type() finds the class (not the module) when decoding the LCM channel type suffix. Verified: entity markers now render as labeled 3D Points in the Rerun world view, positioned at robot odometry coordinates. * CI code cleanup * fix(temporal-memory): use odom:PoseStamped to match Go2 streams Go2 publishes odom: Out[PoseStamped], not odometry: Out[Odometry]. Different name AND type prevented autoconnect from wiring in unitree-go2-temporal-memory blueprint. Changed to odom: In[PoseStamped] which matches the existing convention across all Go2 blueprints. TemporalMemory only reads .position.x/y/z which PoseStamped provides. * fix(temporal-memory): preserve first-detected positions + add insight logging Two fixes: 1. Entity positions: Changed COALESCE order in upsert_entity SQL from COALESCE(excluded.metadata, metadata) to COALESCE(metadata, excluded.metadata). Existing metadata (first detection position) is now preserved on re-sighting. Previously every update overwrote the position to the robot's current location, causing all entities to cluster at one point. 2. Insight logging: Added [temporal-memory] prefixed logger.info() calls for all key events visible in terminal: - Frame/odom counters (every 20 frames) - VLM captions, new entities, entities present, relations - Rolling summaries (300 chars) - Entity marker publish counts - Debug log when no window ready 30 tests pass. * feat(temporal-memory): persistent JSONL, --new-memory flag, better logging Storage changes: - Added persistent JSONL at ~/.local/state/dimos/temporal_memory/ temporal_memory.jsonl that accumulates across runs (raw VLM output + parsed entities). Per-run JSONL still written to run log dir. - --new-memory CLI flag (GlobalConfig) clears both persistent DB and persistent JSONL on startup. Logging improvements: - 'waiting for frames' logged at INFO (was debug) for first 3 polls then every 10th, so users can see the module is alive before frames arrive. - First frame logged immediately (was: after 20 frames). - All [temporal-memory] prefixed logs at INFO level. 30 tests pass. * fix(temporal-memory): move persistent memory to <repo>/memory/temporal_memory/ Robot knowledge is project-specific, not system-wide. The entity graph and accumulated JSONL belong with the project, not in ~/.local/state/. Persistent storage now at: memory/temporal_memory/entity_graph.db (entity graph) memory/temporal_memory/temporal_memory.jsonl (raw VLM dump) Per-run logs stay in ~/.local/state/dimos/logs/<run-id>/. Added memory/temporal_memory/ to .gitignore. Updated README storage docs. * fix(temporal-memory): use CWD for persistent memory, not repo root Path.cwd()/memory/temporal_memory/ instead of dimos.__file__ resolution. Works for both cloned repos and pip installs — memory lives where you run 'dimos run' from, same pattern as .git/ or node_modules/. * fix(temporal-memory): use ~/.local/state/dimos/ for persistent memory XDG state dir — predictable, works for pip install and git clone. No CWD dependency, no repo root detection. Override with db_dir config. * fix(temporal-memory): --new-memory flag now works in forkserver workers GlobalConfig updates happen in the main process AFTER workers fork, so workers never see CLI overrides. Fix: CLI sets DIMOS_NEW_MEMORY=1 env var before fork, TemporalMemory checks both GlobalConfig and env. * fix(temporal-memory): make odom optional, fix test uint8 overflow 1. odom.subscribe() now guarded by transport check — module works without odometry (entities get (0,0,0) positions). Fixes 'NoneType has no attribute subscribe' in integration test. 2. Test image value clamped to 255 (was 50+i*40 overflowing uint8). * fix(temporal-memory): fix integration test — sync, start order, uint8 - Convert async test to sync (no pytest-asyncio needed) - Start consumer (TemporalMemory) before producer (VideoReplay) to avoid race where all frames emit before subscription exists - Clamp test image values to 255 (uint8 overflow at i=6) * fix(temporal-memory): pass --new-memory through blueprint config, not env var Blueprint module is imported lazily by CLI after global_config.update(), so global_config.new_memory is correct at that point. Pass it through TemporalMemoryConfig instead of hacking env vars across fork boundary. * fix(temporal-memory): disable stale scene filter — was dropping nearly all windows stale_scene_threshold=5.0 (mean pixel diff < 5/255) silently skipped almost every window after the first 2. CLIP filtering already handles duplicate frames. Default to 0 (disabled). Added INFO log when skip does fire. * fix(temporal-memory): handle negative replay timestamps in stride check Replay datasets can have negative timestamps (relative to recording start). The stride check 'current - last < stride' always passed when timestamps decreased, blocking all windows after the first 2. Fix: abs() the difference. * fix mypy * chore: retrigger CI --------- Co-authored-by: spomichter <12108168+spomichter@users.noreply.github.com>
Adds --replay-dir flag to select which data/ directory to replay from: dimos --replay run ... # go2_sf_office (default) dimos --replay --replay-dir unitree_go2_bigoffice run ... # big office dataset dimos --replay --replay-dir <any_data_dir> run ... # any dataset Changes: - GlobalConfig: add replay_dir field (default 'go2_sf_office') - ReplayConnection: accept dataset param, use directly as data/ dir name - --replay flag unchanged (backward compatible)
…1521) * fix(rerun): only rate-limit heavy message types (Image, PointCloud2) The blanket per-entity-path rate limiter (PR #1509) was dropping low-frequency but critical messages like navigation Path and PointStamped (click-to-nav). Only rate-limit types with large payloads that actually cause viewer OOM: Image (~1 MB/frame) and PointCloud2 (~600-800 KB/frame). Light messages (Path, Twist, TF, EntityMarkers, etc.) now pass through unthrottled. * Fix import pointcloud2
* feat(drone): add CLI registration and RerunBridge support Phase 1: CLI Registration + Rerun Integration - Create blueprint directory structure (dimos/robot/drone/blueprints/) - Add drone_basic blueprint (DroneConnectionModule + DroneCameraModule + RerunBridge) - Add drone_agentic blueprint (full stack with tracking + agent + web) - Register both blueprints in all_blueprints.py - Replace FoxgloveBridge with RerunBridgeModule - Add replay support via global_config.replay flag - Maintain existing stream names for backward compatibility Blueprints now support: - dimos run drone-basic [--replay] - dimos run drone-agentic [--replay] * feat(drone): remove deprecated main() and mark old Drone class deprecated Phase 2: Module Cleanup - Remove main() function from drone.py (CLI now handles this) - Remove __main__ block - Add deprecation warning to old Drone(Robot) class - Recommend using drone_basic or drone_agentic blueprints instead The old class-based Robot pattern is being phased out in favor of blueprint composition. Existing code using Drone() will still work but should migrate to the new blueprints. * feat(drone): delete deprecated Drone class and add ClockSyncConfigurator - Remove deprecated Drone(Robot) class entirely from drone.py - Delete drone.py file (functionality moved to blueprints) - Update __init__.py to remove Drone export - Add ClockSyncConfigurator to both blueprints (fixes LCM autoconf errors) - Update test_drone.py to skip TestDroneFullIntegration (tested deprecated class) - All remaining tests pass (22 passed, 1 skipped) The Drone class-based pattern is fully deprecated. Use drone_basic or drone_agentic blueprints instead. * fix(drone): remove invalid autoconf=True from LCM() in rerun config - Changed LCM(autoconf=True) to LCM() in both blueprints - Matches Go2 blueprint pattern - Fixes TypeError when loading blueprints - Verified: dimos --replay run drone-basic works (modules deploy successfully) * fix(drone): restore agentic blueprint to match original drone_agentic() signature - Replace conditional viewer logic with direct FoxgloveBridge.blueprint() - Use WebsocketVisModule.blueprint() instead of websocket_vis() alias - Preserve exact module composition: DroneConnectionModule, DroneCameraModule, DroneTrackingModule, WebsocketVisModule, FoxgloveBridge, GoogleMapsSkillContainer, OsmSkill, agent, web_input - Preserve exact remappings: (DroneTrackingModule, video_input → video), (DroneTrackingModule, cmd_vel → movecmd_twist) - Keep function-based _make_drone_agentic() with all original default params - Module-level drone_agentic instance for CLI registry compatibility - Export DRONE_SYSTEM_PROMPT in __all__ * feat(drone): add split Rerun blueprint (camera + 3D view) Opens camera feed automatically on startup instead of requiring manual panel navigation. Horizontal split: Camera (1/3) + 3D world (2/3). * fix(drone): correct Rerun camera origin to world/video DroneCameraModule subscribes on 'video' stream, not 'color_image'. The Rerun entity path is world/video. * fix(google-maps): gracefully handle missing API key GoogleMapsSkillContainer now logs a warning instead of crashing when GOOGLE_MAPS_API_KEY is not set. The module deploys but returns 'not configured' for skill calls. * fix(rerun): convert BGR/BGRA to RGB before sending to Rerun Some Rerun versions don't render BGR color_model correctly, causing blue-tinted video. Convert BGR→RGB and BGRA→RGBA in _format_to_rerun() with numpy channel swap. * Revert "fix(rerun): convert BGR/BGRA to RGB before sending to Rerun" This reverts commit eda0390. * fix(drone): correct replay BGR label and guard OsmSkill RemoteIn - FakeDJIVideoStream: re-tag replay frames from BGR to RGB (GStreamer outputs RGB but Aug 2025 recording used default BGR label) - OsmSkill: guard .subscribe() with hasattr check for RemoteIn compatibility when distributed across workers * fix(drone): use primitive types in move() skill for Pydantic schema compat Replace Vector3 parameter with x/y/z floats so Agent.on_system_modules() can generate JSON schema for the skill. Vector3 is an IsInstanceSchema which Pydantic cannot serialize. * fix(drone): remove n_workers=4, add viewer/replay support to agentic blueprint - Remove wrapper function, flatten to module-level autoconnect - Remove .global_config(n_workers=4) that caused OsmSkill RemoteIn crash - Remove .configurators(ClockSyncConfigurator()) - Add conditional viz: Rerun when --viewer rerun, Foxglove when --viewer foxglove - Add --replay support (connection_string='replay') * fix(drone): rewrite agentic blueprint to match Go2 pattern Clean autoconnect() composition with _vis sub-blueprint for conditional viewer selection. No _modules list, no .insert() hack. * refactor(drone): compose agentic on top of basic blueprint - drone_agentic now imports drone_basic and layers tracking + skills + agent - Removed n_workers=4 and ClockSyncConfigurator from drone_basic - Removed duplicate vis/replay logic from drone_agentic - Removed fill_mode='wireframe' (invalid Rerun FillMode) - Matches Go2 composition pattern * docs(drone): update README for blueprint architecture, fix mypy - Rewrite README for CLI-based usage (dimos run drone-basic/agentic) - Document blueprint composition pattern - Add indoor/outdoor mode, replay, Rerun/Foxglove visualization - Keep RosettaDrone setup verbatim - Add return type annotations to fix mypy no-untyped-def * fix(rerun): fix rate limiter _last_log init and test cleanup - Lazy-init _last_log in _on_message instead of start() so tests can call _on_message without calling start() first - Fix test mock to use spec=RerunConvertible so messages pass through the visual override pipeline - Add bridge.stop() cleanup to prevent thread leak warnings * revert: remove bridge.py rate limiter changes from drone PR These changes (rate limiter + test) were incorrectly added by the subagent in the drone modernization branch. They belong in a separate PR. * botched rebase revert
* pyrealsense2 added as manipulation dependency * renamed manipulation_blueprint to blueprint for name standardization * added back manipulation client for users to interface without using agent cli * updated blueprints to run everything in 1 dimos run command * deleted the unitree-sdk submodule * added Jointstate type for manipulation client methods * fix mypy type checking for manipulation client * fix mymy test * missed blueprint added --------- Co-authored-by: stash <pomichterstash@gmail.com>
…e stale development README (#1524) * docs(cli): add full CLI reference, delete stale dev README - New docs/usage/cli.md: all commands, global options, standalone tools, MCP integration, config precedence, file locations - Delete docs/development/README.md: duplicated main README content - Delete docs/development/dimos_run.md: superseded by cli.md * docs: add system requirements and dependency tier matrix - README: compact hardware table, link to full details - docs/requirements.md: tested configs, all dependency extras, core vs docker tier, platform compatibility notes - Subsumes PR #1402 content with expanded dependency coverage * docs: delete empty todo.md * docs: distinguish Jetson AGX Orin (tested) vs Orin Nano (experimental) * docs: remove core deps and docker extra sections from requirements * docs: add missing psql and dds extras to dependency tiers * docs: README overhaul — beta, CLI/MCP section, zsh quotes, hardware tiers - Alpha Pre-Release → Pre-Release Beta - Arms (Xarm, Piper): experimental → beta - Drones (MAVLink, DJI): experimental → alpha - Delete System Requirements section (moved to docs/requirements.md) - Delete ROS interop section (transports link in Library API) - Add CLI & MCP overview with common commands - Add blueprint composition examples with dimos list - Quote all pip install extras for zsh compatibility - Fix todo.md links → manipulation docs for arms * docs: revert blueprint section to original * docs: update title, trim CLI section, add coding agents blurb * docs: remove headless section from main README (covered in docs/requirements.md) * docs: add featured blueprints section, move coding agent note above installation * docs: featured runfiles with run commands, highlight agent note, move requirements link * docs: move → relative_move per Paul's review, add zsh quotes to requirements.md * docs: use GitHub [!TIP] callout for agent note * docs: show TIP/NOTE/IMPORTANT variants for comparison * docs: use [!IMPORTANT] callout with updated agent wording * docs: install script URL dev → main * docs: move featured runfiles above CLI, 2.4GB → 75MB replay size * docs: trim featured runfiles to 5 key examples * docs: add replay-dir, simulation flags to featured runfiles * docs: add keyboard-teleop-xarm7 to featured runfiles * docs: add ollama blueprint to featured runfiles * docs: write agents readme, delete empty integration guide * docs: DJI SDK → DJI Mavic in hardware table * update agent docs * docs: navigation links to native/index.md, unlink perception stub --------- Co-authored-by: Paul Nechifor <paul@nechifor.net>
…nts (#1525) * feat: add Rerun split layout (Camera | 3D) to Go2 and G1 base blueprints Adds a horizontal split view (camera feed 1/3, 3D world 2/3) to the Rerun viewer for Go2 basic and G1 primitive blueprints, matching the existing drone blueprint pattern. All downstream blueprints inherit the layout automatically. * fix(g1): use MuJoCo computed camera intrinsics instead of hardcoded Go2 values Same fix as PR #1516 for Go2. G1 sim was using hardcoded 1280x720 Go2 real-camera intrinsics while MuJoCo renders at 320x240, causing the projected camera image to appear tiny in Rerun.
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| result = await handle_request(body, request.app.state.skills, request.app.state.rpc_calls) | ||
| if result is None: | ||
| return Response(status_code=204) | ||
| return JSONResponse(result) |
Check warning
Code scanning / CodeQL
Information exposure through an exception Medium
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI about 23 hours ago
In general, to fix information exposure through exceptions, you should avoid returning raw exception messages or stack traces to clients. Instead, log the detailed error server-side (as is already done with logger.exception) and send a generic, non-sensitive message to the client. This preserves debuggability without leaking implementation details.
For this specific case, we should change the except block in _handle_tools_call so that the returned JSON-RPC error text is generic and does not interpolate e. The logging call logger.exception("MCP tool error", tool=name, duration=...) already captures the full exception details, so we can safely remove : {e} from the user-visible message. The best minimal change that preserves existing behavior is:
- Keep the
try/exceptand logging exactly as-is. - Replace
return _jsonrpc_result_text(req_id, f"Error running tool '{name}': {e}")with a generic message such asreturn _jsonrpc_result_text(req_id, f"Error running tool '{name}'.").
No new imports or helper functions are needed; we just change this one return line in dimos/agents/mcp/mcp_server.py.
Suggested changeset
1
dimos/agents/mcp/mcp_server.py
| @@ -118,7 +118,7 @@ | ||
| result = await asyncio.get_event_loop().run_in_executor(None, lambda: rpc_call(**args)) | ||
| except Exception as e: | ||
| logger.exception("MCP tool error", tool=name, duration=f"{time.monotonic() - t0:.3f}s") | ||
| return _jsonrpc_result_text(req_id, f"Error running tool '{name}': {e}") | ||
| return _jsonrpc_result_text(req_id, f"Error running tool '{name}'.") | ||
|
|
||
| duration = f"{time.monotonic() - t0:.3f}s" | ||
|
|
Copilot is powered by AI and may make mistakes. Always verify output.
* fix for macos with pre-set value * mypy fix * fix(autoconf): move route delete to MulticastConfiguratorMacOS.fix() The route delete was in BufferConfiguratorMacOS.fix(), but the corresponding route add lives in MulticastConfiguratorMacOS.fix(). If only buffer settings needed updating, the valid multicast route would be silently deleted without being re-added. --------- Co-authored-by: Summer Yang <summer.yang@outlook.com>
* docs: add demo-camera to featured runfiles * docs: use unitree-go2-agentic-mcp in CLI example * docs: note manipulation extra required for xarm runfile
paul-nechifor
approved these changes
Mar 11, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
9 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Release v0.0.11
82 commits, 10 contributors, 396 files changed.
This release brings a production CLI, MCP tooling, temporal memory, and first-class support for coding agents. Dask has been removed. The entire stack now runs from
dimos runthroughdimos stop.Agent-Native Development
DimOS is now built to be driven by coding agents. Point OpenClaw, Claude Code, or Cursor at AGENTS.md and they can build, run, and debug Dimensional applications using the CLI and MCP interfaces directly.
@skillmethods exposed as HTTP tools. External agents calldimos mcp call relative_move --arg forward=0.5or connect via JSON-RPC.dimos mcp list-tools,dimos mcp call,dimos mcp status,dimos mcp modulesCLI & Daemon
Full process lifecycle — no more Ctrl-C in tmux.
dimos run --daemon— background execution with health checks and run registrydimos stop [--force]— graceful shutdown with SIGTERM → SIGKILL fallbackdimos restart— replays the original CLI argumentsdimos status— PID, blueprint, uptime, MCP portdimos log -f— structured per-run logs with follow, JSON output, filteringdimos show-config— resolved GlobalConfig with source tracingTemporal-Spatial Memory
Robots in physical space ingest hours of video and lidar. Temporal-spatial memory gives them a human-like understanding of the world — causal object relationships, entity tracking through time and physical space, and the ability to answer complex temporal queries:
Who spends the most time in the kitchen? What time on average do I wake up? Which set of switches toggles the main lights? Who was at the office at 9am last Thursday?
Traditional frame-level embeddings (CLIP, ViT) lose temporal context and don't scale beyond a handful of frames. Video transformers are expensive and don't operate in RGB-D. Dimensional agents work with video + lidar natively, tracking entities across hours and days.
Interactive Viewer
Custom Rerun fork (
dimos-viewer) is now the default. Click-to-navigate: click a point in the 3D view → PointStamped → A* planner → robot moves.--viewer rerun|rerun-web|rerun-connect|foxglove|noneDrone Support
Drone blueprints modernized to match Go2 composition pattern.
drone-basicanddrone-agenticwork with replay, Rerun, and the full CLI.More
--robot-ips(feat(robot): Go2 fleet support for multi-robot control #1487)--replay-dir— select dataset, loops by default (feat(replay): named replay datasets via --replay small|big #1519, fix: replay mode loops by default instead of stopping at end #1494)curl -fsSL .../install.sh | bash(feat: interactive install script (DIM-645) #1395)dtopTUI (feat(core): worker resource monitor #1378)Breaking Changes
--viewer-backendrenamed to--viewerrerun-webtorerun(native dimos-viewer)Contributors
@spomichter, @PaulNechifor, @ruthwikdasyam, @summeryang, @MustafaBhadsorawala, @leshy, @sambull, @JeffHykin, @RadientBrain
Contributor License Agreement