[DGX Spark][Build] v1 runtime builder exports OCI archive that Docker rejects

[senthilr-nv]

Investigation Summary

• The documented ARM64 runtime build fails with the default Buildx docker driver because that driver does not support the OCI exporter.
• A task-specific docker-container builder completes the 9.2 GiB OCI export.
• The same NemoClaw command then passes the OCI archive to docker load, which rejects it because the archive has no Docker manifest.json.
• crates/nemoclaw-build/src/runtime.rs selects type=oci at line 92 and runs docker load at line 99.
• No runtime image or deployment resource exists after the failure.

Description

The v1 generic vLLM runtime build cannot complete on a DGX Spark that uses Docker Engine's classic
overlay2 image store.

With the default Buildx docker driver, Buildx rejects the required OCI exporter. With a
docker-container builder, Buildx creates the OCI archive, but NemoClaw supplies that archive to
docker load. Docker expects a Docker archive and reports that manifest.json is absent.

I expected the documented runtime build command to create the OCI archive, load the image into the
selected Docker engine, and return success.

Reproduction Steps

1. Check out v1 commit 0d6f45606eeb3ce4d07121ea63e6760a03594391 on a native Linux ARM64 DGX Spark.
2. Install the versions from versions.json: Rust 1.98.1 and Protobuf compiler 36.1.
3. Run cargo +1.98.1 run -p nemoclaw-build -- runtime runtimes/vllm/build.json with the default Buildx docker driver.
4. Observe OCI exporter is not supported for the docker driver.
5. Create a Buildx builder with the docker-container driver.
6. Set BUILDX_BUILDER to that builder and run the same command.
7. Observe that the OCI export completes and docker load reports invalid archive: does not contain a manifest.json.

Environment

• OS: Ubuntu 24.04, Linux ARM64
• Hardware: NVIDIA DGX Spark, NVIDIA GB10, compute capability 12.1
• NVIDIA driver: 580.126.09
• Docker: 29.1.3, overlay2 storage driver
• Docker Buildx: 0.30.1
• Default builder: BuildKit 0.26.2 with the docker driver
• Task-specific builder: BuildKit 0.32.2 with the docker-container driver
• NemoClaw: v1 commit 0d6f45606eeb3ce4d07121ea63e6760a03594391
• Runtime manifest: runtimes/vllm/build.json

Debug Output

This failure occurs in nemoclaw-build before a NemoClaw deployment exists, so nemoclaw debug
has no deployment state to collect.

The retained archive has this identity:

Size: 9.2 GiB
OCI layout version: 1.0.0
OCI index media type: application/vnd.oci.image.index.v1+json
OCI manifest digest: sha256:3d866a3725b263f9b52876980ca03f4cada927668d3ef428bbc655cb6fbb1fbc
Docker manifest.json: absent
Runtime image tag after failure: absent

The relevant source operations are:

crates/nemoclaw-build/src/runtime.rs:92  type=oci,dest=...
crates/nemoclaw-build/src/runtime.rs:99  docker load -i runtime.tar

Logs

# Default Buildx docker driver
ERROR: failed to build: OCI exporter is not supported for the docker driver.
Switch to a different driver, or turn on the containerd image store, and try again.
Error: "build command failed"

# Buildx docker-container driver
#11 exporting manifest sha256:3d866a3725b263f9b52876980ca03f4cada927668d3ef428bbc655cb6fbb1fbc
#11 sending tarball 14.7s done
#11 DONE 18.4s
invalid archive: does not contain a manifest.json
Error: "build command failed"

Checklist

• I confirmed this bug is reproducible
• I searched existing issues and this is not a duplicate