Change: carry the wire Tensor as the remote-L3 per-argument record

docs/remote-l3-worker-design/buffers-and-transports.md

@@ -141,8 +141,10 @@ Endpoint rules:
   pointer.
 - `RemoteL3Endpoint` requires a sidecar for every tensor payload that crosses
   the remote protocol, including `HOST_INLINE` payloads.
-- Remote TASK frames write `TensorWire.data == 0`; parent virtual
-  addresses never cross the remote protocol.
+- Remote TASK frames carry each argument's `Tensor` verbatim, and that `Tensor`
+  is a `REMOTE_SIDECAR` placeholder: it names the remote backing but describes
+  none of its own. Parent virtual addresses and the parent's own backings never
+  cross the remote protocol.
 - A remote tensor with `child_memory=True` and no sidecar is invalid. Local
   child-memory pointers are meaningful only inside fork/shm topology.
 - The remote session runner translates each `RemoteTensorDesc` into a wire

docs/remote-l3-worker-design/implementation-plan.md

@@ -92,8 +92,7 @@ Status for the local PR #866 cut:
      bounded error payloads.
    - Include tests that reject unknown enum values, non-zero reserved fields,
      and truncated multi-byte fields.
-   - Include tests that reject non-zero `TensorWire.data` in remote
-     TASK frames.
+   - Include tests that reject a remote TASK argument carrying a local backing.
 
 6. Remote callable registry. **Implemented for dispatcher `PYTHON_IMPORT`,
    inner manifest/control `PYTHON_IMPORT`, and inner manifest/control inline
@@ -251,7 +250,7 @@ Status for the local PR #866 cut:
 | Remote import eligibility | Imported peer handle makes only the importer worker eligible. |
 | Remote import dep key | Owner and imported views use the same owner-based TensorMap key. |
 | Raw pointer rejection | Unstaged host pointer fails before slot commit. |
-| Wire data zero | Non-zero remote TASK tensor data is rejected. |
+| Wire backing-free | A remote TASK tensor carrying a local backing or a non-zero `byte_offset` is rejected on encode and on decode. |
 | HOST_INLINE desc | Inline payloads require a descriptor and bounds checks. |
 | Remote buffer copy | Host stages input, remote writes output, host pulls. |
 | Input-only free deferral | Released input buffer survives queued consumers. |

docs/remote-l3-worker-design/protocol.md

@@ -88,24 +88,8 @@ encodes these fields explicitly and rejects drift through decode-time bounds
 checks. The local fork/shm mailbox continues to memcpy the packed `CallConfig`
 POD because it is same-binary IPC, not the cross-host protocol.
 
-`TensorWire v1` encodes tensor metadata explicitly. In remote TASK
-frames, `data` is not a transferable pointer; it is reserved and must be zero.
-
-```text
-data: uint64  # reserved in remote TASK frames; must be 0
-shapes: uint32[MAX_TENSOR_DIMS]
-ndims: uint32
-dtype: uint32
-child_memory: uint8
-reserved: uint8[7]
-```
-
-The wire carries only the contiguous-defining fields; it does **not** carry
-`strides` / `start_offset`, and `decode_tensor` rebuilds them as row-major.
-The remote wire is therefore **contiguous-only**: strided views round-trip
-solely over the local fork/shm mailbox blob (full 128 B `ChipTensor` memcpy).
-`encode_tensor` asserts `is_contiguous && start_offset == 0` so a strided
-tensor fails loudly rather than being silently flattened.
+A remote TASK argument is a `TensorWire`, defined with the rest of the TASK
+payload below.
 
 The session runner decodes these wire records into a local `CallConfig` and a
 wire `TaskArgs` before calling `inner_worker.run()`. Each tensor becomes a
@@ -152,7 +136,8 @@ args: RemoteTaskArgsWire v1
 
 The TASK payload has exactly these top-level fields: callable digest,
 `CallConfigWire`, and `RemoteTaskArgsWire`. `RemoteTaskArgsWire` then carries
-tensor metadata, remote tensor descriptors, scalars, and optional inline bytes.
+the argument tensors, remote tensor descriptors, scalars, and optional inline
+bytes.
 
 TASK frames carry the raw digest from the submitted parent-facing
 `CallableHandle.hashid`. Parent-side dependency inference has already consumed
@@ -165,7 +150,7 @@ the sidecar descriptors captured at submit time to name, on the session runner,
 the local backings the wire `TaskArgs` views.
 
 The current `RemoteL3Endpoint` implementation builds this payload from
-`TaskSlotState`, zeros every tensor metadata `data` field, and submits the
+`TaskSlotState`, carrying each argument's `Tensor` verbatim, and submits the
 encoded frame through `RemoteL3Transport`. The simulation session runner
 resolves the digest in its dispatcher registry, materializes the sidecar
 descriptors, and calls `inner_worker.run()`.
@@ -175,22 +160,28 @@ descriptors, and calls `inner_worker.run()`.
 ```text
 tensor_count: uint32
 scalar_count: uint32
-tensor_metadata: TensorWire[tensor_count]
+tensors: TensorWire[tensor_count]
 remote_desc: OptionalRemoteTensorDescWire[tensor_count]
 scalars: uint64[scalar_count]
 inline_payload_bytes_len: uint32
 inline_payload_bytes: uint8[inline_payload_bytes_len]
 ```
 
-For each tensor index, exactly one of these is true:
-
-- `remote_desc[i]` is present and names a remote buffer, imported peer buffer,
-  UB mapping, or allowed small `HOST_INLINE` payload.
-- The tensor is metadata-only and has no data pointer and no remote descriptor.
-
-`tensor_metadata[i].data` must be zero in both cases. Bare host pointers are
-rejected for remote endpoints unless an explicit staging API has produced a
-remote handle and sidecar descriptor.
+`TensorWire` is the wire `Tensor` of `src/common/task_interface/buffer.h`: the
+embedded `BufferDescriptor` (identity nonce, `buffer_id`, `generation`,
+`address_space`, `access`, `backend_kind`, `nbytes`, `owner_worker_path_id`, a
+length-delimited backend body) followed by the view (`byte_offset`, `ndims`,
+`ndims`-many `shapes` and `strides`, `dtype`). Shapes and strides travel as
+`ndims`-many entries, so the slots past `ndims` that `validate_tensor` requires
+zero are never on the wire.
+
+Every argument's `remote_desc[i]` is present and names a remote buffer,
+imported peer buffer, UB mapping, or allowed small `HOST_INLINE` payload. That
+sidecar is the sole authority for the argument's backing, so `tensors[i]` must
+carry no backing of its own: its `backend_kind` is `REMOTE_SIDECAR` and its
+`byte_offset` is zero, both rejected otherwise on encode and on decode. Bare
+host pointers are rejected for remote endpoints unless an explicit staging API
+has produced a remote handle and sidecar descriptor.
 
 `HOST_INLINE` is for small payloads that should travel inside the TASK frame.
 It still requires a `RemoteTensorDescWire` with `address_space=HOST_INLINE`;
@@ -221,14 +212,13 @@ RemoteTensorDescWire:
 
 Rules:
 
-- `ChipTensor` remains the L2 ABI. The session runner translates descriptors
-  into wire `Tensor` args immediately before `inner_worker.run()`; the L2 leaf
-  under it materializes those into `ChipTensor` as it does on every path.
-- When a descriptor is present, the incoming `TensorWire.data` is
-  reserved and must be zero. The session runner derives the executable local
-  address only from the validated descriptor and its live buffer/import
-  registry.
-- Metadata-only tensors also require `TensorWire.data == 0`.
+- `ChipTensor` remains the L2 ABI. The session runner rebuilds each incoming
+  `TensorWire`'s view over the local backing its descriptor names, immediately
+  before `inner_worker.run()`; the L2 leaf under it materializes those into
+  `ChipTensor` as it does on every path.
+- The incoming `TensorWire` carries no address and no local backing. The
+  session runner derives the executable local address only from the validated
+  descriptor and its live buffer/import registry.
 - Parent-side dependency keys use a stable logical start-address key derived at
   submit time:
   `(address_kind, owner_worker_id, buffer_id, generation, offset)`.
@@ -769,7 +759,8 @@ The frame codec must reject:
 - descriptor offsets outside the referenced handle;
 - `HOST_INLINE` payload offsets or lengths outside the inline byte arena;
 - non-`HOST_INLINE` descriptors with non-zero inline payload lengths;
-- non-zero `TensorWire.data` in remote TASK frames;
+- a remote TASK argument that carries a local backing (any `backend_kind` other
+  than `REMOTE_SIDECAR`) or a non-zero `TensorWire.byte_offset`;
 - stale generations;
 - unknown control names or control versions;
 - completion sequence mismatch;

python/simpler/buffer.py

@@ -60,6 +60,7 @@
     "intern_worker_path",
     "mint_owner_instance_id",
     "re_export",
+    "remote_backing_identity",
     "remote_sidecar_tensor",
     "worker_path_for_id",
     "wrap_device_malloc",
@@ -254,6 +255,20 @@ def re_export(source: BufferDescriptor) -> Buffer:
     )
 
 
+def remote_backing_identity(owner_worker_id: int, buffer_id: int, generation: int) -> CanonicalIdentity:
+    """The canonical identity of a backing that lives on another machine's worker.
+
+    A remote owner's ``owner_instance_id`` never crosses the remote L3 wire, so the nonce is the
+    owning worker's id instead. This is the single rule for naming a remote backing: the submitting
+    L4's ``REMOTE_SIDECAR`` placeholder and the importing session runner both derive from it, so one
+    remote backing carries one identity on both sides of the hop.
+    """
+    oid = int(owner_worker_id).to_bytes(OWNER_INSTANCE_ID_BYTES, "little")
+    # A HOST_INLINE placeholder has no backing and so no generation of its own; 0 is the reserved
+    # "uninitialized" value a decoder rejects, so it carries the initial generation instead.
+    return CanonicalIdentity(oid, int(buffer_id), int(generation) or 1)
+
+
 def remote_sidecar_tensor(
     shapes: tuple[int, ...],
     dtype: int,
@@ -271,13 +286,11 @@ def remote_sidecar_tensor(
     descriptor rides in the per-task RemoteTaskArgsSidecar). The identity encodes the remote buffer
     (``owner_worker_id`` folded into the opaque nonce, plus ``buffer_id`` / ``generation``) so
     dependency inference and routing stay stable across the hop.
+
+    This placeholder is what the remote L3 wire carries verbatim as the task's per-argument record.
     """
-    oid = int(owner_worker_id).to_bytes(OWNER_INSTANCE_ID_BYTES, "little")
-    # A HOST_INLINE placeholder has no backing and so no generation of its own; 0 is the reserved
-    # "uninitialized" value a decoder rejects, so the placeholder carries the initial generation.
-    identity = CanonicalIdentity(oid, buffer_id, int(generation) or 1)
     descriptor = BufferDescriptor(
-        identity=identity,
+        identity=remote_backing_identity(owner_worker_id, buffer_id, generation),
         owner_worker_path_id=intern_worker_path(f"remote/{owner_worker_id}"),
         address_space=address_space,
         access=AccessMode.READWRITE,

python/simpler/remote_l3_protocol.py

@@ -15,17 +15,30 @@
 import struct
 from dataclasses import dataclass
 
-from .task_interface import MAX_TENSOR_DIMS, CallConfig, ChipTensor, DataType
-
-# 2: CallConfig lost its block_dim field — a run always takes the whole
-# device, so the payload is one int32 shorter than v1's.
-PROTOCOL_VERSION = 2
+from .buffer import (
+    OWNER_INSTANCE_ID_BYTES,
+    AccessMode,
+    AddressSpace,
+    BackendKind,
+    BufferDescriptor,
+    CanonicalIdentity,
+    Tensor,
+)
+from .task_interface import MAX_TENSOR_DIMS, CallConfig, DataType
+
+# 3: a TASK's per-argument record is the self-describing wire ``Tensor`` — the embedded
+# BufferDescriptor plus the strided view. Both ends of a run come from one ``pip install``,
+# so this constant is a mismatch alarm at the frame header, not a dual-decode selector.
+PROTOCOL_VERSION = 3
 MAX_FRAME_PAYLOAD_BYTES = 16 * 1024 * 1024
 MAX_STRING_BYTES = 1024
 MAX_ERROR_BYTES = 4096
 MAX_TENSORS = 4096
 MAX_SCALARS = 4096
 MAX_INLINE_PAYLOAD_BYTES = 1024 * 1024
+# Mirrors DESC_MAX_BYTES in src/common/task_interface/buffer.h. ``BufferDescriptor`` construction
+# re-checks it, so this bound only makes an over-long body fail at the field that carries it.
+MAX_BUFFER_DESCRIPTOR_BODY_BYTES = 32
 MAX_TRANSPORT_PROFILE_BYTES = 128
 MAX_TRANSPORT_DESCRIPTOR_BYTES = 4096
 HOST_TCP_TRANSPORT_PROFILE = "host_tcp"
@@ -146,7 +159,7 @@ class RemoteTensorSidecar:
 
 @dataclass(frozen=True)
 class RemoteTaskArgsWire:
-    tensor_metadata: tuple[ChipTensor, ...]
+    tensors: tuple[Tensor, ...]
     remote_desc: tuple[RemoteTensorSidecar, ...]
     scalars: tuple[int, ...]
     inline_payload: bytes
@@ -431,22 +444,47 @@ def decode_call_config(reader: _Reader) -> CallConfig:
     return cfg
 
 
-def decode_tensor(reader: _Reader) -> ChipTensor:
-    data = reader.u64()
-    if data != 0:
-        raise ValueError("remote_wire: remote TASK tensor data must be zero")
-    shapes = [reader.u32() for _ in range(MAX_TENSOR_DIMS)]
+def decode_tensor(reader: _Reader) -> Tensor:
+    """The wire ``Tensor`` a remote TASK carries per argument: embedded descriptor plus view.
+
+    Shapes and strides are ``ndims``-many, so the slots past ``ndims`` that ``validate_tensor``
+    requires zero are never on the wire. ``Tensor`` construction runs that validator, which is what
+    keeps this decode behind the same gate as every other Tensor trust boundary.
+    """
+    owner_instance_id = reader.raw(OWNER_INSTANCE_ID_BYTES, "buffer identity nonce")
+    buffer_id = reader.u64()
+    generation = reader.u32()
+    address_space = reader.u8()
+    access = reader.u8()
+    backend_kind = reader.u8()
+    # An arg bound for a remote worker has no local backing: the authoritative descriptor of its
+    # backing is the per-argument RemoteTensorDesc sidecar.
+    if backend_kind != int(BackendKind.REMOTE_SIDECAR):
+        raise ValueError("remote_wire: a remote TASK tensor must carry no local backing")
+    nbytes = reader.u64()
+    owner_worker_path_id = reader.u32()
+    body = reader.blob(MAX_BUFFER_DESCRIPTOR_BODY_BYTES, "BufferDescriptor.body")
+    byte_offset = reader.u64()
+    # The sidecar's own offset is where the view sits in the backing, so the record's view spans
+    # exactly the backing its descriptor advertises.
+    if byte_offset != 0:
+        raise ValueError("remote_wire: a remote TASK tensor must carry no byte_offset")
     ndims = reader.u32()
     if ndims == 0 or ndims > MAX_TENSOR_DIMS:
         raise ValueError("remote_wire: tensor ndims out of range")
+    shapes = tuple(reader.u32() for _ in range(ndims))
+    strides = tuple(reader.u32() for _ in range(ndims))
     dtype = DataType(reader.u32())
-    child_memory = reader.u8()
-    if child_memory not in (0, 1):
-        raise ValueError("remote_wire: tensor child_memory must be 0 or 1")
-    for _ in range(7):
-        if reader.u8() != 0:
-            raise ValueError("remote_wire: ChipTensor reserved bytes must be zero")
-    return ChipTensor.make(0, tuple(shapes[:ndims]), dtype, bool(child_memory))
+    descriptor = BufferDescriptor(
+        identity=CanonicalIdentity(owner_instance_id, buffer_id, generation),
+        address_space=AddressSpace(address_space),
+        access=AccessMode(access),
+        backend_kind=BackendKind(backend_kind),
+        nbytes=nbytes,
+        body=body,
+        owner_worker_path_id=owner_worker_path_id,
+    )
+    return Tensor(buffer=descriptor, byte_offset=byte_offset, shapes=shapes, strides=strides, dtype=dtype)
 
 
 def decode_remote_tensor_desc(reader: _Reader) -> RemoteTensorDesc:

python/simpler/remote_l3_session.py

@@ -34,15 +34,14 @@
 from typing import Any, Callable
 
 from .buffer import (
-    OWNER_INSTANCE_ID_BYTES,
     AccessMode,
     AddressSpace,
     BackendKind,
     Buffer,
-    CanonicalIdentity,
     create_host_shared_buffer,
     intern_worker_path,
     mint_owner_instance_id,
+    remote_backing_identity,
 )
 from .callable_identity import (
     CallableHandle,
@@ -558,13 +557,13 @@ def _buffer_key(buffer_id: int, generation: int) -> tuple[int, int]:
 def _import_wire_buffer(export_desc: Any, shm_name: str, base: int) -> Buffer:
     """The wire ``Buffer`` naming an imported backing, whose owner is another worker's process.
 
-    The owner's ``owner_instance_id`` never crosses the remote wire, so the identity is rebuilt from
-    the export triple the same way the submitting L4's ``REMOTE_SIDECAR`` placeholder builds it —
-    the importing runner and the submitter therefore name one remote backing with one identity.
+    The backing is named by ``remote_backing_identity`` — the one rule the submitting L4's
+    ``REMOTE_SIDECAR`` placeholder also uses — so one remote backing carries one identity on both
+    sides of the hop. It is minted here, at IMPORT_BUFFER, because a local address for an imported
+    shm exists from then on and every later task over it must name the same backing.
     """
-    oid = int(export_desc.owner_worker_id).to_bytes(OWNER_INSTANCE_ID_BYTES, "little")
     return Buffer(
-        identity=CanonicalIdentity(oid, int(export_desc.buffer_id), int(export_desc.generation)),
+        identity=remote_backing_identity(export_desc.owner_worker_id, export_desc.buffer_id, export_desc.generation),
         owner_worker_path_id=intern_worker_path(f"remote/{int(export_desc.owner_worker_id)}"),
         address_space=AddressSpace.HOST,
         access=AccessMode.READWRITE,
@@ -585,31 +584,40 @@ def _materialize_task_args(
 ) -> tuple[TaskArgs, list[Buffer]]:
     """The wire ``TaskArgs`` a remote TASK's orchestration function receives.
 
-    Every element is a ``Tensor`` over a backing this runner holds, so an orchestration function
-    forwards its own args to a chip child exactly as one at any other level does. The returned
-    ``Buffer`` list is the session-scoped backings minted for this task's HOST_INLINE payloads; they
-    live no longer than the run and the caller closes them once it returns.
+    The wire carries each argument as the submitter's own ``REMOTE_SIDECAR`` ``Tensor``: its view
+    (shapes, strides, dtype) is taken verbatim, and the sidecar names the local backing that view is
+    rebuilt over. Every element is therefore a ``Tensor`` over a backing this runner holds, so an
+    orchestration function forwards its own args to a chip child exactly as one at any other level
+    does. The returned ``Buffer`` list is the session-scoped backings minted for this task's
+    HOST_INLINE payloads; they live no longer than the run and the caller closes them once it
+    returns.
     """
-    if len(args.remote_desc) != len(args.tensor_metadata):
-        raise ValueError("remote TASK descriptor count does not match tensor metadata count")
+    if len(args.remote_desc) != len(args.tensors):
+        raise ValueError("remote TASK descriptor count does not match tensor count")
     task_args = TaskArgs()
     inline_backings: list[Buffer] = []
 
     try:
-        for tensor, sidecar in zip(args.tensor_metadata, args.remote_desc):
+        for tensor, sidecar in zip(args.tensors, args.remote_desc):
             if not sidecar.present:
                 raise ValueError("remote TASK tensor payload requires a RemoteTensorRef sidecar")
             desc = sidecar.desc
             if desc is None:
                 raise ValueError("remote TASK descriptor is marked present but missing")
-            if desc.nbytes != tensor.nbytes():
-                raise ValueError("remote TASK descriptor nbytes does not match tensor metadata")
+            if desc.nbytes != tensor.buffer.nbytes:
+                raise ValueError("remote TASK descriptor nbytes does not match the tensor's backing")
+            # The sidecar is the sole authority for where the view sits in the backing, so the
+            # placeholder's own descriptor spans exactly the view and its byte_offset is zero.
+            if tensor.byte_offset != 0:
+                raise ValueError("remote TASK tensor must carry no byte_offset of its own")
             if desc.address_space == RemoteAddressSpace.HOST_INLINE:
                 backing, byte_offset = _materialize_inline_payload(args, desc, mint_inline_buffer)
                 inline_backings.append(backing)
             else:
                 backing, byte_offset = _resolve_session_backing(desc, buffers, worker_id)
-            task_args.add_tensor(backing.tensor(tuple(tensor.shapes), tensor.dtype, byte_offset=byte_offset))
+            task_args.add_tensor(
+                backing.tensor(tensor.shapes, tensor.dtype, strides=tensor.strides, byte_offset=byte_offset)
+            )
     except BaseException:
         # A rejected arg leaves the caller no list to release, so the backings minted for the args
         # that did decode are released here.