Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,8 @@ add_custom_target(build_package_sim
if(SKBUILD_MODE)
install(DIRECTORY ${CMAKE_SOURCE_DIR}/src/
DESTINATION simpler_setup/_assets/src)
install(DIRECTORY ${CMAKE_SOURCE_DIR}/cmake/
DESTINATION simpler_setup/_assets/cmake)
install(DIRECTORY ${CMAKE_SOURCE_DIR}/build/lib/
DESTINATION simpler_setup/_assets/build/lib
OPTIONAL
Expand Down
5 changes: 3 additions & 2 deletions docs/python-packaging.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ simpler_setup/ ← test framework + build/runtime assembly (simpler_s
paged_attention.py attention reference (used by multiple paged_attention tests)
_assets/ (wheel-only, populated by CMake install)
src/ source tree mirror
cmake/ shared modules used by runtime compilation
build/lib/ pre-built per-arch/platform/runtime .so/.o

_task_interface.*.so nanobind extension at site-packages root
Expand Down Expand Up @@ -69,10 +70,10 @@ NPU hardware (`a2a3`/`a5` with CANN toolkit).

`simpler_setup.environment.PROJECT_ROOT` auto-detects between:

- **Wheel install**: `simpler_setup/_assets/` exists → `PROJECT_ROOT = .../site-packages/simpler_setup/_assets`. The wheel's bundled `_assets/src/` and `_assets/build/lib/` provide everything needed at runtime.
- **Wheel install**: `simpler_setup/_assets/` exists → `PROJECT_ROOT = .../site-packages/simpler_setup/_assets`. The wheel's bundled `_assets/src/`, `_assets/cmake/`, and `_assets/build/lib/` provide everything needed at runtime.
- **Source tree / editable install**: `_assets/` doesn't exist → `PROJECT_ROOT = repo root`. Live `src/` and `build/lib/` are used.

Anything that needs to find `src/`, `build/lib/`, or `build/cache/` MUST go through `simpler_setup.environment.PROJECT_ROOT` — never `Path(__file__).parent.parent...`.
Anything that needs to find `src/`, `cmake/`, `build/lib/`, or `build/cache/` MUST go through `simpler_setup.environment.PROJECT_ROOT` — never `Path(__file__).parent.parent...`.

## Import rules

Expand Down
6 changes: 6 additions & 0 deletions examples/workers/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ workers/
global_tload_mixed_l3/ # Global CommDomain build + cross-machine peer TLOAD on both ranks
compute_then_tload_mixed_l3/ # compute round on both L2s, then peer TLOAD through the same domain
global_tload_mpirun_l3/ # one mpirun launches an L3 rank per machine; MPI descriptor exchange
vector_add_mpi_direct_l3/ # L4 joins MPI rank 0 and controls both real L3 ranks through direct P2P
```

Why no `tensormap_and_ringbuffer/` layer? Because every example here hard-codes
Expand Down Expand Up @@ -67,6 +68,11 @@ parent owns a single `mpirun` that launches an L3 rank on each machine
example's README for its extra prerequisites (`mpirun` + `mpi4py` on both
machines).

`vector_add_mpi_direct_l3` is the direct-MPI alternative: one supervisor
launches L4 as MPI rank 0 and the two real L3 workers as ranks 1 and 2. It also
needs `mpirun` and `mpi4py` on both machines, but task/control traffic is P2P
between L4 and each L3 rather than passing through the PR2 group mailbox.

### What a new L4 example needs

```text
Expand Down
68 changes: 68 additions & 0 deletions examples/workers/l4/vector_add_mpi_direct_l3/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
# L4 direct MPI L3 vector add

This example launches one static MPI world across two machines: rank 0 owns
L4, rank 1 owns a real L3 on the L4 machine, and rank 2 owns a real L3 on the
peer. L4 sends SLR3 task and control frames directly to both L3 ranks, and
each L3 dispatches to its local L2 workers.

```text
machine A machine B

MPI rank 0 / L4 -------- direct MPI --------> MPI rank 2 / real L3
|
+------------- direct MPI -----------> MPI rank 1 / real L3
```

The example validates direct MPI startup and teardown, remote memory copy,
L3-to-L2 vector-add execution, result checking, and Global CommDomain
lifecycle. The vector-add kernel does not perform a peer `TLOAD`.

## Prerequisites beyond the sibling example

- `mpirun`/`mpiexec` and `mpi4py` must be installed on **both** machines and
built against the **same** MPI implementation.
- `mpirun` executes one identical command line on both machines, so `--python`
must name an interpreter path valid on BOTH. Point it at a per-machine
launcher script installed at one shared absolute path; each machine's copy
sources CANN, enters that machine's checkout, and execs its `.venv` Python:

```bash
cat > /tmp/simpler-mpi-python <<'EOF'
#!/usr/bin/env bash
source /usr/local/Ascend/cann/set_env.sh
cd /path/to/this/machines/simpler
exec /path/to/this/machines/simpler/.venv/bin/python "$@"
EOF
chmod +x /tmp/simpler-mpi-python
```

## Run on the L4 parent

`192.0.2.10` / `192.0.2.20` are documentation placeholders. Hosts must be
numeric IPs; the local host is where ranks 0 and 1 run:

```bash
source .venv/bin/activate
python -m examples.workers.l4.vector_add_mpi_direct_l3.main \
--local-host 192.0.2.10 --remote-host 192.0.2.20 \
--python /tmp/simpler-mpi-python \
--local-devices 0,1 --remote-devices 0,1 \
--mpirun-path "$(command -v mpiexec)" \
--launcher-family mpich
```

Success requires the process to return status 0 after printing:

```text
vector_add_mpi_direct_l3 passed
```

Any failed vector validation exits non-zero.

## Running it in CI

The network1 job's `network1-stage` action writes the per-machine launcher on both
machines at one shared path and exports it as `NETWORK1_MPI_PYTHON`; the
`test_vector_add_mpi_direct_l3.py` wrapper reads it together with `NETWORK1_LOCAL_IP`
and the standard network1 fixtures, and skips when `mpirun`, `mpi4py`, or either
variable is absent.
9 changes: 9 additions & 0 deletions examples/workers/l4/vector_add_mpi_direct_l3/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# Copyright (c) PyPTO Contributors.
# This program is free software, you can redistribute it and/or modify it under the terms and conditions of
# CANN Open Software License Agreement Version 2.0 (the "License").
# Please refer to the License for details. You may not use this file except in compliance with the License.
# THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED,
# INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE.
# See LICENSE in the root of the software repository for the full text of the License.
# -----------------------------------------------------------------------------------------------------------
"""L4-to-L3 direct MPI vector-add example."""
Loading
Loading