Skip to content

feat(kfdtest): Cu mask enablement PR AILIKFD-28 - #9738

Open
asierrag-amd wants to merge 5 commits into
developfrom
users/asierrag-amd/develop_wip
Open

feat(kfdtest): Cu mask enablement PR AILIKFD-28#9738
asierrag-amd wants to merge 5 commits into
developfrom
users/asierrag-amd/develop_wip

Conversation

@asierrag-amd

@asierrag-amd asierrag-amd commented Aug 5, 2026

Copy link
Copy Markdown 
BasicCuMaskingLinear assumes mask bit i selects the i-th CU on the engine
that runs the dispatch, which breaks on a multi-XCC node (e.g. gfx12.1.0
in SPX mode) in two ways.

The driver spreads the linear mask across XCCs (bit i -> XCC i % NumXcc),
but the PM4 queue runs on a single XCC, so most enabled bits never grow
the executing XCC's CU count. Stride the mask by NumXcc and cap the sweep
at ActiveCU / NumXcc so every enabled CU lands on XCC0.

Within that XCC the mask fills round-robin across shader engines, so
parallelism grows only in steps of SEsPerXcc. Compare against this
SE-balanced effective CU count instead of the raw count.

For single-XCC parts (NumXcc == 1) this reduces to the original
contiguous mask, so behavior is unchanged.

@asierrag-amd
kfdtest: make BasicCuMaskingLinear XCC-aware
9d15080 
BasicCuMaskingLinear assumes mask bit i selects the i-th CU on the engine
that runs the dispatch, which breaks on a multi-XCC node (e.g. gfx12.1.0
in SPX mode) in two ways.

The driver spreads the linear mask across XCCs (bit i -> XCC i % NumXcc),
but the PM4 queue runs on a single XCC, so most enabled bits never grow
the executing XCC's CU count. Stride the mask by NumXcc and cap the sweep
at ActiveCU / NumXcc so every enabled CU lands on XCC0.

Within that XCC the mask fills round-robin across shader engines, so
parallelism grows only in steps of SEsPerXcc. Compare against this
SE-balanced effective CU count instead of the raw count.

For single-XCC parts (NumXcc == 1) this reduces to the original
contiguous mask, so behavior is unchanged.
@asierrag-amd
kfdtest: make ExtendedCuMasking gfx12.1 multi-XCC aware
57c98ed 
On gfx12.1 the CU mask ABI is one bit per WGP, interleaved across XCCs
(unit u of XCC x is at x + u*numXcc), and a PM4 dispatch runs on a single
XCC. The old flat-SE, two-bit-per-WGP flow mis-maps every mask on such
parts (e.g. gfx1250), marking all CUs outside XCC0 inactive.

Add a per-XCC path (gated on NumXcc > 1 within FAMILY_GFX12):

- Let a PM4 queue select its target XCC via BaseQueue::Create; existing
  callers are unchanged.
- Extend mask_config_t with numXcc/targetXcc; numXcc <= 1 keeps the
  legacy encoding so pre-gfx12.1 parts are unaffected.
- Make setCUMask/adjustMask honour the one-bit-per-WGP interleaved
  encoding and drive/validate each XCC's WGPs in isolation.
- Treat KFD topology as WGP-granular (drop the CU<->WGP factor of two)
  so the per-XCC WGP count and mask size are not halved.

Also fix SE readback on gfx12.5+, where HW_ID1 no longer carries SE: the
shader reads it from MSG_RTN_GET_SE_AID_ID. Rewrite adjustMaskXcc to
mirror KFD's interleaved enumeration and compaction of inactive WGPs,
handling asymmetric WGP-per-SA layouts (e.g. SA0=9, SA1=8 on gfx1250).

Signed-off-by: Alex Sierra <alex.sierra@amd.com>
@asierrag-amd
kfdtest: enable CU masking tests on gfx1250
3353918 
Run the CU masking tests on gfx1250 now that they are multi-XCC aware
and expected to pass on this ASIC:

- Drop KFDQMTest.ExtendedCuMasking from the gfx1250 exclude profile.
- Split the CU masking tests (BasicCuMaskingLinear, BasicCuMaskingEven)
  out of PERMANENT_BLACKLIST_ALL_ASICS into CU_MASKING_BLACKLIST, and add
  a BLACKLIST_ALL_ASICS_NO_CU_MASK base that omits them. gfx1250 builds
  on this base so the tests are executed, while all other profiles keep
  excluding them via BLACKLIST_ALL_ASICS (unchanged behavior).
@therock-pr-bot

therock-pr-bot Bot commented Aug 5, 2026

Copy link
Copy Markdown

❌ PR Check — Action Required

Check Status Details
📝 PR Description ❌ Fail Error: PR description must reference a JIRA ID, ISSUE ID, or a GitHub closing keyword.
Expected: include a JIRA ID / ISSUE ID line (separator : or -, or omitted; value may be a JIRA key, a number with/without #, or a link), OR a closing keyword + issue reference. Accepted examples:
• JIRA ID : TESTAUTO-6039
• JIRA ID - #330
• JIRA ID #330
• JIRA ID (on separate line)
ROCM-25757
• ISSUE ID : TESTUTO-3334
• ISSUE ID #3334
• ISSUE ID - TESTAUTO-3433
• ISSUE ID (on separate line)
AIRUNTIME-2352
• ISSUE ID : https://github.com/<org_name>/<repo_name>/issues/1234
• Closes #10
• Fixes octo-org/octo-repo#100
• Resolves: #123
• #123
• https://github.com/<org_name>/<repo_name>/issues/123
Current: no valid JIRA/ISSUE/closing-keyword reference found
⛔ Forbidden Files ✅ Pass —
🧪 Unit Test ⚠️ Warning Error: Source/code files changed without an accompanying unit test.
Expected: add at least one test file named like test_<name>.py / test_<name>.cpp (or <name>_test.*).
Current: code file(s) changed: projects/rocr-runtime/libhsakmt/tests/kfdtest/src/BaseQueue.cpp, projects/rocr-runtime/libhsakmt/tests/kfdtest/src/BaseQueue.hpp, projects/rocr-runtime/libhsakmt/tests/kfdtest/src/KFDQMTest.cpp, projects/rocr-runtime/libhsakmt/tests/kfdtest/src/KFDQMTest.hpp, projects/rocr-runtime/libhsakmt/tests/kfdtest/src/ShaderStore.cpp; no test file found
🔎 pre-commit ⏳ Pending ⏳ Still running…
🚫 Draft PR 🔜 To Be Enabled —
🚩 Feature Flag 🔜 To Be Enabled —
📊 Code Coverage 🔜 To Be Enabled —

⚠️ 1 policy check(s) failed. Please address the issues above before this PR can be Reviewed.

🚫 Please fix the failed policies

  • ❌ PR Description

The Not ready to Review label was added to this PR. Once all policies pass, the label is removed automatically.

📖 Need help? See the Policy FAQ for details on every check and how to fix failures.

🙋 Wish to Override Policy?

@therock-pr-bot

therock-pr-bot Bot commented Aug 5, 2026

Copy link
Copy Markdown

🚫 Please fix the failed policies before requesting reviews.

The following policy checks failed:

  • ❌ PR Description

The Not ready to Review label has been added to this PR.
Once all policies pass, the label will be removed automatically.

@dayatsin-amd dayatsin-amd changed the title kfdtest Cu mask enablement feat(kfdtest): Cu mask enablement Aug 5, 2026
@dayatsin-amd

dayatsin-amd commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

In the future please use this format for the PR descriptions:
feat(kfdtest): Cu mask enablement

@asierrag-amd asierrag-amd changed the title feat(kfdtest): Cu mask enablement feat(kfdtest): Cu mask enablement PR AILIKFD-28 Aug 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@kentrussell kentrussell Awaiting requested review from kentrussell kentrussell is a code owner
@xiaogang-chen-amd xiaogang-chen-amd Awaiting requested review from xiaogang-chen-amd xiaogang-chen-amd is a code owner
@dayatsin-amd dayatsin-amd Awaiting requested review from dayatsin-amd

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

Not ready to Review organization: ROCm project: rocr-runtime

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@asierrag-amd @dayatsin-amd