[FEATURE] remote policy with server/client #310

kanghui0204 · 2025-12-30T09:30:30Z

Hi Team,

I added a remote server/client mechanism for IsaacLab-Arena that allows the IsaacLab environment and the policy model environment to run separately. Alex and I wrote a design document(design doc) for this feature, which includes our discussions about the design. Below is an overview of this PR.

1. Functionality

Previously, the IsaacLab-Arena pipeline ran entirely in a single process:

IsaacLab env → Env policy class (e.g., Gr00tClosedloopPolicy) → local policy model (e.g., Gr00tPolicy), all in one environment.

This PR adds remote policy/server–client support and decouples the env policy class from the local policy model. The new pipeline becomes:

Client: IsaacLab env → Env policy class (e.g., Gr00tClosedloopPolicy)

Server: local policy model (e.g., Gr00tPolicy)

The client and server exchange observations and actions via sockets.

The server/client implementation lives in isaaclab_arena/remote_policy inside IsaacLab-Arena. Users can copy this directory and import it on the server side directly, without needing to install it as a separate package.

2. How to use it

On the client side, you still use isaaclab_arena/examples/policy_runner.py, but you now pass a few extra arguments. Example:

python isaaclab_arena/examples/policy_runner.py \
  --policy_type gr00t_closedloop \
  --policy_deployment remote \  # where the policy model is deployed; 'local' is the original ISAACLab-Arena flow, 'remote' enables the client-side remote policy
  --remote_host 127.0.0.1 \     # server policy IP
  --remote_port 5555 \          # server policy listening port
  --remote_api_token API_TOKEN_123 \  # API token for server/client communication
  --policy_config_yaml_path isaaclab_arena_gr00t/gr1_manip_gr00t_closedloop_config.yaml \
  --num_steps 2000 \
  --num_envs 10 \
  --enable_cameras \
  --headless \
  --remote_kill_on_exit \  # whether to kill the server policy when the client exits; by default the server stays alive, enabling reuse; if enabled, the server is shut down when the client finishes
  gr1_open_microwave \
  --embodiment gr1_joint

On the server side, there is a Python entry point isaaclab_arena.remote_policy.remote_policy_server_runner.py. You specify host/port/etc., select the policy type, and provide the policy config file:

python -m isaaclab_arena.remote_policy.remote_policy_server_runner \
  --host 0.0.0.0 \
  --port 5555 \
  --api_token API_TOKEN_123 \
  --timeout_ms 5000 \
  --policy_type gr00t_closedloop \
  --policy_config_yaml_path /absolute/path/to/gr1_manip_gr00t_closedloop_config.yaml

3. Current example

Right now there is a working example for GR00T. On a given machine, the steps are:

Start the server

export MODELS_DIR=/path/to/your/gr00t/models
bash ./docker/run_gr00t_server.sh

This uses the following defaults inside the script:

host: 0.0.0.0
port: 5555
api_token: API_TOKEN_123
timeout_ms: 5000
policy_type: gr00t_closedloop
policy_config_yaml_path: /workspace/isaaclab_arena_gr00t/gr1_manip_gr00t_closedloop_config.yaml

If needed, you can override these via command-line flags, for example:

export MODELS_DIR=/path/to/your/gr00t/models

bash ./docker/run_gr00t_server.sh \
  --port 6000 \
  --api_token MY_TOKEN \
  --policy_config_yaml_path /workspace/isaaclab_arena_gr00t/my_custom_config.yaml

Start the client
Set up the IsaacLab Docker container following
IsaacLab Arena docker documentation(no GR00T installation is needed inside this container).

Inside the container, run:

python isaaclab_arena/examples/policy_runner.py \
  --policy_type gr00t_closedloop \
  --policy_deployment remote \
  --remote_host 127.0.0.1 \
  --remote_port 5555 \
  --remote_api_token API_TOKEN_123 \
  --policy_config_yaml_path isaaclab_arena_gr00t/gr1_manip_gr00t_closedloop_config.yaml \
  --num_steps 2000 \
  --num_envs 10 \
  --enable_cameras \
  --headless \
  --remote_kill_on_exit \
  gr1_open_microwave \
  --embodiment gr1_joint

With this setup, you can obtain results for the GR1 open microwave task using a remote GR00T policy server.
and I can get the result:
Metrics: {'success_rate': 0.57, 'door_moved_rate': 0.935, 'num_episodes': 200}

4. What remains to be done? Future work

At the moment, documentation for this feature has not been updated. I’d like the team to review this PR first and confirm that the usage and interface look reasonable.

Planned future work includes:

Support for the new VLN task.
Improved communication efficiency. Currently, the server and client communicate over sockets. We can explore more efficient transports and/or observation compression to reduce communication latency.

alexmillane

Thank you for putting this together.

Most of my comments are minor syntactic things that we can easily address.

I have one major comment on the design. At the moment, the server-client split point leaves half of gr00t policy running on the client and half (the model itself) running on the server. To me it would make sense to move the split point such that the whole policy is launched on the server. That way, in the remote inference case, we remove all gr00t details from the client side. It also simplifies the server side - the server just launches the same policy (called Gr00tClosedloopPolicy) that is launched in the local case.

This diagram explains (a simplified version) of the current design and the alternative that I'm proposing.

alexmillane · 2026-01-05T01:35:02Z

docker/setup/install_gr00t_deps_wo_isaac.sh

+#!/bin/bash
+set -euo pipefail
+
+# Script to install GR00T policy dependencies
+# This script is called from the GR00T server Dockerfile
+
+: "${GROOT_DEPS_GROUP:=base}"
+: "${WORKDIR:=/workspace}"
+
+echo "Installing GR00T with dependency group: $GROOT_DEPS_GROUP"
+
+# CUDA environment variables for GR00T installation.
+# In the PyTorch base image, CUDA is already configured, so we only
+# set variables if CUDA_HOME exists.
+if [ -d "/usr/local/cuda" ]; then
+    export CUDA_HOME=${CUDA_HOME:-/usr/local/cuda}
+    export PATH=${CUDA_HOME}/bin:${PATH}
+    export LD_LIBRARY_PATH=${CUDA_HOME}/lib64:${LD_LIBRARY_PATH:-}
+fi
+
+echo "CUDA environment variables:"
+echo "CUDA_HOME=${CUDA_HOME:-unset}"
+echo "PATH=$PATH"
+echo "LD_LIBRARY_PATH=${LD_LIBRARY_PATH:-unset}"
+
+# Install system-level media libraries (no sudo in container)
+echo "Installing system-level media libraries..."
+apt-get update && apt-get install -y ffmpeg && rm -rf /var/lib/apt/lists/*
+
+# Upgrade packaging tools
+echo "Upgrading packaging tools..."
+python -m pip install --upgrade setuptools packaging wheel
+
+# Install Isaac-GR00T with the specified dependency group
+echo "Installing Isaac-GR00T with dependency group: $GROOT_DEPS_GROUP"
+python -m pip install --no-build-isolation --use-pep517 \
+    -e "${WORKDIR}/submodules/Isaac-GR00T/[$GROOT_DEPS_GROUP]"
+
+# Install flash-attn (optional, keep same version as Arena Dockerfile)
+echo "Installing flash-attn..."
+python -m pip install --no-build-isolation --use-pep517 flash-attn==2.7.1.post4 || \
+    echo "flash-attn install failed, continue without it"
+
+echo "GR00T dependencies installation completed successfully"


Thanks for putting this together.

This script seems to differ only very slightly from the install_gr00t_deps.sh. Is the only difference the python command used? I.e. python vs /isaac-sim/python.sh.

Could we combine these two scripts into a single script that takes an argument(s)?

I remove this to install_gr00t_deps.sh and add a arguement on install_gr00t_deps.s

alexmillane · 2026-01-05T01:45:16Z

docker/run_gr00t_server.sh

+# REQUIRED: host models directory (must be set by user)
+if [[ -z "${MODELS_DIR:-}" ]]; then
+  echo "ERROR: MODELS_DIR is not set."
+  echo "Please export MODELS_DIR to your host models directory, e.g.:"
+  echo "  export MODELS_DIR=/path/to/your/models"
+  echo "Then run:"
+  echo "  bash ./docker/run_gr00t_server.sh"
+  exit 1
+fi


Suggestion to used the same thing used in run_docker.sh:

Models are by default expected on the host at $HOME/models but this can be changed with the -d flag to the script.

See default and the optional override

please check again ，I remove this part and add similar code as run_docker.sh

alexmillane · 2026-01-05T01:54:03Z

docker/run_gr00t_server.sh

+  --timeout_ms "${TIMEOUT_MS}" \
+  --policy_type "${POLICY_TYPE}" \
+  --policy_config_yaml_path "${POLICY_CONFIG_YAML_PATH}"
+


Not strictly related to this MR, but I'm wondering if we should move all the gr00t related stuff. I.e. all the gr00t docker stuff and the gr00t related tests out of the core framework and into the isaaclab_arena_gr00t package. Actually this is basically certainly a good idea. Ideally the core framework would make no mention of gr00t.

It would probably make sense to do this before merging this MR. So we can get everything gr00t related in the right package, before expanding what we can do there.

I have now made the change I suggested above. All gr00t-related code lives in isaaclab_arena_gr00t

alexmillane · 2026-01-05T02:03:18Z

isaaclab_arena/examples/policy_runner_cli.py

            num_steps = policy.get_trajectory_length(policy.get_trajectory_index())

    elif args.policy_type == "gr00t_closedloop":
+        from pathlib import Path


Suggestion to move to the top of the file.

We have some imports not at the top of the file if they require special dependencies that are conditionally not required. But prefer to put imports at the top.

alexmillane · 2026-01-05T02:05:20Z

isaaclab_arena/examples/policy_runner_cli.py

+    remote_group.add_argument(
+        "--remote_api_token",
+        type=str,
+        default=None,
+        help="Optional API token for remote policy server.",
+    )


Why is an API token (optionally) required? Suggestion to expand the help string here.

I add a detail description ，please check

alexmillane · 2026-01-05T02:55:53Z

isaaclab_arena/remote_policy/policy_client.py

+        # if options is not None:
+        #     payload["options"] = options


Intentional? options is currently unused.

remove options，right now it is not used. When I was developing this, I considered that if some special information needed to be transmitted, an extra options field might be necessary. Since it is not needed at the moment, I have removed it.

alexmillane · 2026-01-05T02:59:38Z

isaaclab_arena/remote_policy/policy_client.py

+        if isinstance(resp, dict) and "action" in resp:
+            return resp["action"]
+        return resp


Why have this optional unwrapping? Can we assume one form of the response? Either the action dict, or a dict containing the action dict? Is there a reason to accept both?

also removed

alexmillane · 2026-01-05T03:08:11Z

isaaclab_arena_gr00t/policy_config.py

+        if not Path(self.model_path).exists():
+            warnings.warn(
+                "[GR00TConfig] model_path does not exist: "
+                f"{self.model_path}. No model checkpoint was found. "
+                "If this is the client side of a remote policy, this warning can be ignored. "
+                "However, if you are running a local policy or the server side of a remote policy, "
+                "the program will fail to load the model and cannot run correctly."
+            )


Suggestion to remove this warning altogether. From the message the warning will certainly fire in cases where nothing is wrong. That's pretty confusing for the user, I feel, so I'd suggest that we remove this check here. Perhaps we could move the check to a more appropriate place? For example in the local policy where we know we need a valid model path.

OK ，I remove it now，we can add it once we have fully separated the policy and the client.

alexmillane · 2026-01-05T03:31:07Z

isaaclab_arena/remote_policy/remote_policy_server_runner.py

+POLICY_REGISTRY: dict[str, str] = {
+    # policy_type: "module_path:ClassName"
+    "gr00t_closedloop": "isaaclab_arena_gr00t.gr00t_remote_policy:Gr00tRemoteModelPolicy",
+}


I'm thinking we should generalize and use our registry class used for Assets, Devices, and Retargetters for Policies too.

But I agree with the idea here: let's have a policy registry to allow users to register policies, to free the core code from directly depending on policy code. I think that this will be critical to getting policy (currently gr00t) code out of the core framework.

I add a registry class in remote_policy folder .please check

Thank you for doing that. Unfortunately we've double-done the work 🥲 . I added a new registry as part of moving the gr00t code out of the core package in #316 . I would suggest that you use my registry (which utilizes the same machinery we use for registering assets etc.)

alexmillane · 2026-01-05T03:36:39Z

isaaclab_arena/remote_policy/remote_policy_config.py

Suggestion to rename to remote_policy_config.py

kanghui0204 force-pushed the socket_for_policy branch from b65f8cb to f542ea6 Compare December 30, 2025 13:10

alexmillane changed the base branch from release/0.1.1 to main January 4, 2026 19:18

alexmillane changed the base branch from main to release/0.1.1 January 4, 2026 19:20

alexmillane reviewed Jan 5, 2026

View reviewed changes

kanghui0204 force-pushed the socket_for_policy branch from f542ea6 to 698c2bf Compare January 5, 2026 06:25

kanghui0204 changed the base branch from release/0.1.1 to main January 5, 2026 06:28

add socket code for policy

f9296d8

kanghui0204 force-pushed the socket_for_policy branch from 698c2bf to f9296d8 Compare January 6, 2026 06:23

[FEATURE] remote policy with server/client #310

Are you sure you want to change the base?

[FEATURE] remote policy with server/client #310

Uh oh!

Conversation

kanghui0204 commented Dec 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

1. Functionality

2. How to use it

3. Current example

4. What remains to be done? Future work

Uh oh!

alexmillane left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

kanghui0204 commented Dec 30, 2025 •

edited

Loading