Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
427 changes: 427 additions & 0 deletions docs/agents/deploy-agents.mdx

Large diffs are not rendered by default.

6 changes: 4 additions & 2 deletions docs/agents/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -84,8 +84,8 @@ To run a workflow YAML directly without registering it on the platform, pass `--
A platform-managed agent consists of three components:

1. **The agent entity.** `nemo agents create` stores the workflow YAML as an entity in the workspace. The same configuration can be redeployed, evaluated, or optimized without re-registering it.
1. **The deployment controller.** `nemo agents deploy` passes the stored config to the Agents service controller, which launches a `nat start fastapi` process for it, assigns a port, watches its health, and tears it down on `nemo agents undeploy`.
1. **The Agents gateway.** Client requests reach the agent at `/apis/agents/v2/workspaces/<workspace>/agents/<name>/-/<path>`. The gateway resolves the agent to its current running deployment and proxies the request, including streaming responses. From a client's perspective, the agent is an OpenAI-compatible endpoint owned by NeMo Platform.
1. **The deployment controller.** `nemo agents deploy` passes the stored config to the Agents service controller. By default it launches a `nat start fastapi` process, assigns a port, watches its health, and tears it down on `nemo agents undeploy`. With `--mode docker` or `--mode k8s` it instead runs the agent as a durable container through the deployments plugin — see [Deploy Agents](/documentation/agents/deploy-agents).
1. **The Agents gateway.** The gateway resolves the agent to its current running deployment and proxies client requests to it, including streaming responses. From a client's perspective, the agent is an OpenAI-compatible endpoint owned by NeMo Platform.

Model traffic from inside the agent process routes back through the Inference Gateway, which resolves model entity names to upstream providers and supplies their credentials. This is why agent configs do not carry `base_url` or `api_key` values — the deployment injects the gateway URL automatically, and the gateway looks up the rest.

Expand All @@ -109,6 +109,8 @@ undeploying the candidate.

## Common Tasks

- [Deploy Agents](/documentation/agents/deploy-agents): run an agent as a local subprocess or as a
durable container on Docker or Kubernetes, and invoke it through the Agents gateway.
- [Optimize Agents](/documentation/agents/optimize-agents): analyze deployed agents for model routing,
skill, prompt, and new-model opportunities.
- [Secure Agents](/documentation/agents/secure-agents): check guardrail coverage and scan recent
Expand Down
2 changes: 2 additions & 0 deletions docs/fern/versions/latest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -203,6 +203,8 @@ navigation:
- section: Agents
path: ../../agents/index.mdx
contents:
- page: Deploy Agents
path: ../../agents/deploy-agents.mdx
- page: Optimize Agents
path: ../../agents/optimization.mdx
- page: Secure Agents
Expand Down
1 change: 1 addition & 0 deletions packages/nemo_platform/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -222,6 +222,7 @@ models-service = [
# Generated from [tool.bundle-package]; do not edit by hand.
nemo-agents-example-calculator = [
"nvidia-nat-core>=1.8.0,<1.9",
"nvidia-nat-langchain>=1.8.0,<1.9",
]

# Generated from [tool.bundle-package]; do not edit by hand.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,9 @@ description = "NAT ``calculator`` function group for the nemo-agents calculator
requires-python = ">=3.11"
dependencies = [
"nvidia-nat-core>=1.8.0,<1.9",
# Provides the ``react_agent`` workflow and the ``openai`` LLM the agent uses.
# Required so a packaged container image (nemo agents package) can run the agent.
"nvidia-nat-langchain>=1.8.0,<1.9",
]
version = "0.0.0"

Expand Down
9 changes: 7 additions & 2 deletions plugins/nemo-agents/src/nemo_agents_plugin/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -90,8 +90,13 @@ class DeploymentsRunnerConfig(BaseModel):
),
)
config_mount_path: str = Field(
default="/config/agent.yaml",
description="Path inside the container where the NAT workflow config is mounted.",
default="/workspace/config.yaml",
description=(
"Path inside the container where the NAT workflow config is placed. Must sit under "
"the image's writable WORKDIR (/workspace) so docker mode, which materializes the "
"config as the non-root container user, can write it; k8s mounts it read-only there "
"via a ConfigMap subPath. Matches the image's NAT_CONFIG_FILE convention."
),
)


Expand Down
31 changes: 28 additions & 3 deletions plugins/nemo-agents/src/nemo_agents_plugin/sdk.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,10 @@
nemo.agents.delete("calculator")

# Deployment lifecycle
dep = nemo.agents.deployments.create(agent="calculator")
dep = nemo.agents.deployments.create(agent="calculator") # subprocess
dep = nemo.agents.deployments.create(
agent="calculator", deployment_mode="docker", image="calculator:local"
)
deps = nemo.agents.deployments.list()
dep = nemo.agents.deployments.get("calculator-a1b2")
nemo.agents.deployments.delete("calculator-a1b2")
Expand Down Expand Up @@ -211,12 +214,34 @@ def create(
*,
agent: str,
name: str | None = None,
deployment_mode: str = "subprocess",
image: str | None = None,
workspace: str = _DEFAULT_WORKSPACE,
) -> dict[str, Any]:
"""Create a deployment for *agent*."""
payload: dict[str, Any] = {"agent": agent}
"""Create a deployment for *agent*.

Args:
agent: Name of the agent to deploy.
name: Deployment name (auto-generated if omitted).
deployment_mode: Runtime backend — ``"subprocess"`` (default),
``"docker"``, or ``"k8s"``. Container modes run the agent as a
durable container through the deployments plugin and require a
configured executor.
image: Container image for ``docker``/``k8s`` modes. Falls back to
``agents.deployments.default_image`` when omitted. Rejected in
``subprocess`` mode.
workspace: Target workspace.

Returns:
The created deployment as a dict.
"""
if image and deployment_mode == "subprocess":
raise ValueError("image requires deployment_mode='docker' or 'k8s'.")
payload: dict[str, Any] = {"agent": agent, "deployment_mode": deployment_mode}
if name:
payload["name"] = name
if image:
payload["image"] = image
return self._parent._post(f"/v2/workspaces/{workspace}/deployments", payload)

def list(self, workspace: str = _DEFAULT_WORKSPACE) -> List[dict[str, Any]]:
Expand Down
17 changes: 12 additions & 5 deletions plugins/nemo-agents/tests/unit/test_runner_deployments.py
Original file line number Diff line number Diff line change
Expand Up @@ -65,14 +65,21 @@ def test_executor_for_mode_prefers_mode_specific() -> None:
assert executor_for_mode(cfg, "k8s") == "k8s-exec"


def test_config_mount_path_default_is_under_writable_workspace() -> None:
# Docker mode materializes the config as the non-root container user, so the
# default must live under the image's writable WORKDIR (/workspace); a
# root-level path like /config is not writable and crash-loops the container.
assert DeploymentsRunnerConfig().config_mount_path.startswith("/workspace/")


def test_build_deployment_config_always_single_container() -> None:
cfg = build_deployment_config(
name="hello-dep",
workspace="default",
image="nat-runtime:latest",
port=8000,
nat_config={"llms": {"nim": {"_type": "nim"}}},
config_mount_path="/config/agent.yaml",
config_mount_path="/workspace/config.yaml",
mode="docker",
gateway_base_url="http://host.docker.internal:8080",
)
Expand All @@ -86,7 +93,7 @@ def test_build_deployment_config_always_single_container() -> None:
assert container.readiness_probe is not None
assert cfg.init_containers == []
assert len(cfg.config_files) == 1
assert cfg.config_files[0].path == "/config/agent.yaml"
assert cfg.config_files[0].path == "/workspace/config.yaml"
loaded = yaml.safe_load(cfg.config_files[0].content)
assert loaded["llms"]["nim"]["_type"] == "nim"

Expand All @@ -98,7 +105,7 @@ def test_build_deployment_config_k8s_uses_nat_entrypoint() -> None:
image="nat-runtime:latest",
port=8000,
nat_config={},
config_mount_path="/config/agent.yaml",
config_mount_path="/workspace/config.yaml",
mode="k8s",
gateway_base_url="http://nmp:8080",
)
Expand All @@ -114,7 +121,7 @@ def test_build_deployment_config_k8s_option_b_when_image_set() -> None:
image="nat-runtime:latest",
port=8000,
nat_config={},
config_mount_path="/config/agent.yaml",
config_mount_path="/workspace/config.yaml",
mode="k8s",
gateway_base_url="http://nmp:8080",
plugin_wheels_init_image="busybox:1.36",
Expand All @@ -131,7 +138,7 @@ def test_build_deployment_config_docker_never_emits_init_containers() -> None:
image="nat-runtime:latest",
port=8000,
nat_config={},
config_mount_path="/config/agent.yaml",
config_mount_path="/workspace/config.yaml",
mode="docker",
gateway_base_url="http://host.docker.internal:8080",
plugin_wheels_init_image="busybox:1.36",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -202,10 +202,16 @@ async def create_deployment(
"environment": env_dict(container_spec),
**restart_policy_kwargs(config.restart_policy, config.backoff_limit),
}
# Mirror Kubernetes semantics (see the k8s compiler): a container spec's
# ``command`` is the entrypoint override and ``args`` is the command
# (CMD). Map them onto docker's distinct ``entrypoint`` / ``command``
# kwargs so a driven container overrides the image's baked-in ENTRYPOINT
# instead of appending to it. Conflating both into docker ``command``
# leaves the image ENTRYPOINT in force and silently drops the spec.
if container_spec.command:
run_kwargs["command"] = container_spec.command
run_kwargs["entrypoint"] = list(container_spec.command)
if container_spec.args:
run_kwargs["command"] = list(container_spec.command) + list(container_spec.args)
run_kwargs["command"] = list(container_spec.args)

volume_bindings = build_volume_bindings(workspace, merged_volume_mounts(config, container_spec))
if volume_bindings:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,35 @@ async def test_create_deployment_starts_container(
mock_entities.get.assert_awaited()


@pytest.mark.asyncio
async def test_create_deployment_maps_command_to_entrypoint(
docker_backend: DockerDeploymentBackend,
mock_entities: AsyncMock,
mock_docker_client: MagicMock,
) -> None:
"""A spec's ``command`` overrides the image ENTRYPOINT; ``args`` is the CMD.

This mirrors Kubernetes semantics and is required so a driven container
(e.g. a packaged agent that bakes its own ``ENTRYPOINT``) runs the
platform-supplied command instead of the image default.
"""
mock_entities.get.return_value = sample_config() # command=["echo"], args=["hello"]
mock_docker_client.containers.get.side_effect = NotFound("missing")
mock_docker_client.containers.run.return_value = MagicMock(id="abc123")

await docker_backend.create_deployment(
workspace="default",
name="srv",
config_name="cfg1",
labels={"managed-by": MANAGED_BY_LABEL},
backend_config={},
)

_, run_kwargs = mock_docker_client.containers.run.call_args
assert run_kwargs["entrypoint"] == ["echo"]
assert run_kwargs["command"] == ["hello"]


@pytest.mark.asyncio
async def test_read_status_ready_when_running_without_probe(
docker_backend: DockerDeploymentBackend,
Expand Down
Loading
Loading