diff --git a/.claude/skills/test-all-device/SKILL.md b/.claude/skills/test-all-device/SKILL.md index 52559a7ff2..dde20bd042 100644 --- a/.claude/skills/test-all-device/SKILL.md +++ b/.claude/skills/test-all-device/SKILL.md @@ -16,15 +16,34 @@ Detection / isolation procedures referenced below live in `--pto-session-timeout`. 4. **Select a device range** (§C, range ≤4) — or, when wrapping in `task-submit`, let it pick via `--device auto --device-num `. -5. **Run through `task-submit`** (§E). The underlying command: +5. **Read the marker selector out of the same job** you took the timeout from. + `st-onboard-a2a3` is two pytest passes, not one: the sweep deselects + `-m "not sdma"` and a later step runs `-m sdma`. `st-onboard-a5` has no + marker filter. Take the expression from `ci.yml` rather than assuming, so + this skill cannot drift from the job it reproduces. +6. **Run through `task-submit`** (§E). The underlying command, on a platform + whose job carries the filter: ```bash - pytest examples tests/st --platform --device \ - --pto-session-timeout -v + pytest examples tests/st -m "not sdma" --platform \ + --device --pto-session-timeout -v ``` + Then the quarantined pass, after it — never at the same time, since the + point of the split is that no fault-injection case meets a device that has + already provisioned SDMA: + + ```bash + pytest examples tests/st -m sdma --platform \ + --device <2 devs or $TASK_DEVICE> --pto-session-timeout -v + ``` + + Dropping either pass changes what you tested: a single flat sweep is not + what CI runs and reports failures CI never sees + ([`../testing/SKILL.md`](../testing/SKILL.md), issue #1425). + Parallelism is auto-driven by `--device`: on hardware, one in-flight subprocess per device (`--max-parallel auto` = `len(--device)`); see `docs/testing.md` for the full reuse hierarchy. -6. Report the results summary (pass/fail counts per task). -7. If any tests fail, show the relevant error output and which device failed. +7. Report the results summary (pass/fail counts per task), across both passes. +8. If any tests fail, show the relevant error output and which device failed. diff --git a/.claude/skills/test-runtime-device/SKILL.md b/.claude/skills/test-runtime-device/SKILL.md index cd6bb55a06..6b3a026ca5 100644 --- a/.claude/skills/test-runtime-device/SKILL.md +++ b/.claude/skills/test-runtime-device/SKILL.md @@ -18,15 +18,30 @@ Detection / isolation procedures referenced below live in `--pto-session-timeout`. 5. **Select a device range** (§C, range ≤4) — or, when wrapping in `task-submit`, let it pick via `--device auto --device-num `. -6. **Run through `task-submit`** (§E). The underlying command: +6. **Read the marker selector out of the same job** you took the timeout from. + `st-onboard-a2a3` is two pytest passes: the sweep deselects `-m "not sdma"` + and a later step runs `-m sdma`. `st-onboard-a5` has no marker filter. Both + quarantined tests are `tensormap_and_ringbuffer`, so the second pass is only + needed when `$ARGUMENTS` names that runtime. +7. **Run through `task-submit`** (§E). The underlying command: ```bash - pytest examples tests/st --platform --runtime $ARGUMENTS \ + pytest examples tests/st -m "not sdma" --platform --runtime $ARGUMENTS \ --device \ --pto-session-timeout -v ``` + Then, for `tensormap_and_ringbuffer` on a platform whose job carries the + filter, the quarantined pass afterwards — never concurrently, since the + split exists so that no fault-injection case meets a device that has already + provisioned SDMA ([`../testing/SKILL.md`](../testing/SKILL.md), issue #1425): + + ```bash + pytest examples tests/st -m sdma --platform --runtime $ARGUMENTS \ + --device <2 devs or $TASK_DEVICE> --pto-session-timeout -v + ``` + Hardware parallelism is auto-driven by `--device` (one subprocess per device); no extra flag needed. -7. Report the results summary (pass/fail counts per task). -8. If any tests fail, show the relevant error output and which device failed. +8. Report the results summary (pass/fail counts per task), across both passes. +9. If any tests fail, show the relevant error output and which device failed. diff --git a/docs/ci.md b/docs/ci.md index 14a916b836..3149386401 100644 --- a/docs/ci.md +++ b/docs/ci.md @@ -82,10 +82,12 @@ benefit — device bin-packing for L3, xdist fanout for L2, and a shared `ChipWorker` per `(runtime, device)`: ```bash -# Recommended CI invocation -pytest examples tests/st --platform a2a3 --device 4-7 -x +# Recommended CI invocation — a2a3 deselects the SDMA marker, as the job does, +# and runs it as a second pass afterwards +pytest examples tests/st -m "not sdma" --platform a2a3 --device 4-7 -x +pytest examples tests/st -m sdma --platform a2a3 --device 4-5 -x -# Same for a5 +# Same for a5, which has no marker filter pytest examples tests/st --platform a5 --device 0-7 -x ```