Docs: realign ci.md with the gates it describes - #1617
Merged
Conversation
|
Important Review skippedAuto incremental reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
📝 WalkthroughWalkthroughThe CI documentation adds profiling and example-corpus detection outputs. It expands change-detection axes and documents separate SDMA execution for the ChangesCI scheduling documentation
Estimated code review effort: 1 (Trivial) | ~5 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/ci.md`:
- Line 48: Update the st-onboard-a2a3 workflow description in docs/ci.md to
state that SDMA is a separate sequential pytest step, with ARM64 devices
acquired through task-submit and x86_64 devices acquired directly via
${DEVICE_RANGE}. Clarify that the general and SDMA steps reuse the
architecture-specific device range rather than implying a disjoint allocation.
- Line 132: Update the documentation in the `detect-changes` section to
correctly describe empty PR diff handling. The current wording implies all flags
including examples_only are set to true, but actually only the architecture and
test category flags are true while examples_only remains false. Remove or
clarify the "every arch and category flag true" phrasing to accurately represent
that examples_only is false in the empty diff case, preventing the false
implication that packaging and profiling would run under that path.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ChaoWao
force-pushed
the
docs/ci-gating-drift
branch
from
July 31, 2026 13:01
ad27dc9 to
b2088ec
Compare
`docs/ci.md` is the page a reader consults to learn what CI actually runs, so
each of these read as a statement of fact and each was false.
- **`profiling-flags-smoke` was missing from the job tree entirely.** It exists,
runs on `ubuntu-latest`, and is gated `(a2a3_changed || a5_changed) &&
!examples_only`. The prose two sections down already discussed it, so the tree
was the only place that denied it.
- **`detect-changes` was listed as emitting four outputs, not five.**
`examples_only` was absent.
- **"computes three flags (`a2a3_changed`, `a5_changed`, `non_code_only`)"** —
the two bullets immediately following it describe the test-category and
example-corpus axes, so the paragraph contradicted its own neighbours. It now
names all four axes and keeps the load-bearing claim, that every one of them
derives from a single `NON_CODE` set.
- **`st-onboard-a2a3` was described as a bare `pytest examples tests/st`.** The
sweep deselects `-m "not sdma"` and a separate step runs `-m sdma`. A reader
mirroring the job from this table reproduced the sweep without the isolation
and would not learn why their SDMA case behaved differently. `st-onboard-a5`
has no marker filter and its row was already correct.
The scheduling-constraints list gains the SDMA split as a constraint in its own
right, and separates the part that always holds from the part that does not.
Ordering holds on both host arches — the SDMA step is second, so no
fault-injection case can meet a device that has already provisioned SDMA.
Device disjointness holds only on aarch64, where the step takes its own
`task-submit --device auto`; the x86_64 branch has no `task-submit` and reuses
`${DEVICE_RANGE}`, so ordering is its only separation. The bullet names hw-native-sys#1425
as the condition for dropping the split, and states that nothing outside
`st-onboard-a2a3` filters on the marker, so a local `pytest examples tests/st`
is not silently narrower.
The empty-diff paragraph now lists `examples_only=false` alongside the flags the
guard sets. It was scoped to "every arch and category flag", which excluded
`examples_only` correctly but left the reader to guess — and it is the value
that makes "running the full matrix" include packaging and the profiling smoke.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This was referenced Jul 31, 2026
ChaoWao
added a commit
that referenced
this pull request
Aug 2, 2026
`st-onboard-a2a3` has been two pytest invocations since #1609 — the sweep with `-m "not sdma"` and a later step with `-m sdma`, so that no fault-injection case meets a device that has already provisioned SDMA (#1425). Three places that claim to reproduce that job still issued one flat sweep, which is the invocation `.claude/skills/testing/SKILL.md` warns "is *not* what CI runs and will report failures that CI never sees". - **`test-all-device`** describes itself as running "the full hardware (onboard) CI pipeline" and already reads `--pto-session-timeout` out of the `st-onboard-<platform>` job. It now reads the marker expression from the same place, and runs the second pass after the first. Taking both from `ci.yml` rather than hardcoding them is what keeps the skill from drifting again. - **`test-runtime-device`** likewise. Both quarantined tests are `tensormap_and_ringbuffer`, so the second pass is noted as needed only when that runtime is selected. - **`docs/ci.md`'s "Recommended CI invocation"** block, which #1617 missed while correcting the job table three sections above it. The a5 line is unchanged — `st-onboard-a5` carries no marker filter. Left alone: the recipes that describe running the tests locally rather than mirroring a job (`README.md`, `docs/testing.md`, `docs/user/reference/cli.md`, `examples/README.md`, the `SUBMIT_BY_CLUSTER.md` trio). Whether a local sweep should exclude SDMA by default is a question about the documented default, not about matching CI, and `testing/SKILL.md` already tells anyone who trips over it what happened.
3 tasks
4 tasks
ChaoWao
added a commit
that referenced
this pull request
Aug 2, 2026
Everywhere a doc told the reader to run the full a2a3 scene-test sweep on real hardware, it gave a bare `pytest examples tests/st --platform a2a3` — the exact invocation `.claude/skills/testing/SKILL.md` warns "is *not* what CI runs and will report failures that CI never sees": it mixes the SDMA cases with the `sweep's` `aicore_op_timeout` fault-injection test, which is the collision the dedicated SDMA step exists to prevent (#1425, and the ~306 s fault teardown an SDMA-provisioned device pays). All nine sites now carry `-m "not sdma"` and name the marker, so a local sweep reproduces CI instead of surprising it: - `README.md`, `docs/user/reference/cli.md`, `examples/README.md` — hardware scan lines - `docs/testing.md` — both hardware scan sites; the first gains the separate `-m sdma` pass after it, with the reason - `docs/troubleshooting/a2a3-507899-aicpu-shared-so-fault.md` — the repro scan - `src/{a2a3,a5}/runtime/{tensormap_and_ringbuffer,host_build_graph}/docs/ SUBMIT_BY_CLUSTER.md` — the three final-validation scans Sim lines and single-directory invocations are untouched — the marker only matters where SDMA and fault injection can meet. `st-onboard-a5` carries no marker filter, so a5 scan lines stay as they were. The two `ci.yml` comments claiming the SDMA step runs "on its own dedicated device(s)" get the same correction applied to the docs in #1617: device disjointness holds only on aarch64, where the step takes `task-submit --device auto`; the x86_64 branch shares `${DEVICE_RANGE}` with the sweep and ordering is the separation. Left deferred since #1617 because touching `ci.yml` books the whole matrix — this change does that anyway.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
docs/ci.mdhad drifted fromci.ymlin four places, all introduced by #1601 / #1607 / #1609 and all reading as statements of fact:profiling-flags-smokerow at allubuntu-latest,(a2a3_changed || a5_changed) && !examples_onlydetect-changesoutputs 4 flagsexamples_onlywas missingst-onboard-a2a3runs a barepytest examples tests/st-m "not sdma"plus a dedicated-m sdmastep on its own devicesThe last one is the one that misleads in practice: it is the table a reader uses to mirror a job locally, and following it reproduces the sweep without the SDMA isolation.
Also adds the SDMA device split to the scheduling-constraints list — it is a scheduling constraint in its own right — naming #1425 as the condition for removing it, and noting that nothing outside
st-onboard-a2a3filters on the marker so a localpytest examples tests/stis not silently narrower.No behaviour change;
docs/only.Testing
if:inci.ymlagainst the doc's claims: 9 gated jobs match, and the only ungated ones arepre-commitanddetect-changes, exactly as documentedst-onboard-a5genuinely has no-mfilter, so its row was already correct and is untouchedpytest examples tests/stmentions across the repo — they are local-run instructions, correctly unfiltered, and deliberately left alonenon_code_only, so every job exceptpre-commitanddocsshould reportskipping— a live check of the gates it documents