Skip to content

external-builds/pytorch: add a torch-only ROCm ASAN build mode - #7313

Draft
sa-faizal wants to merge 2 commits into
ROCm:mainfrom
sa-faizal:draft/asan-wheels-phase2-pytorch
Draft

external-builds/pytorch: add a torch-only ROCm ASAN build mode#7313
sa-faizal wants to merge 2 commits into
ROCm:mainfrom
sa-faizal:draft/asan-wheels-phase2-pytorch

Conversation

@sa-faizal

Copy link
Copy Markdown
Contributor

Stack: 2 of 2, after #7312.

For the Phase 2-only review diff, compare draft/asan-wheels-phase1-rocm...draft/asan-wheels-phase2-pytorch.

Summary

Add the integrated Phase 2 production-builder mode for a torch-only ROCm ASAN wheel built from the isolated Phase 1 package set:

  • validate the local manifest, exact coherent ROCm ASAN version, bootstrap frontend, and single gfx942:xnack+ target;
  • install the SDK offline with --no-index, --no-build-isolation, exact versioning, and the selector's device extra;
  • select packaged ROCm Clang, bitcode, and the shared ASAN runtime and configure shared-ASAN/frame-pointer flags;
  • disable CMake's optional module scanner while preserving caller CMAKE_ARGS;
  • derive a collision-resistant Torch suffix from the selected ROCm candidate;
  • keep the prebuilt ASAN AOTriton path enabled without building a separate Triton wheel; and
  • scope LD_PRELOAD to the post-install Torch sanity subprocess only.

ASAN mode opts into the generic portable-RPATH contract from #7210; its parser/archive-gate implementation is deliberately not duplicated here.

Why this is a draft

  • the reusable builder still contains pilot policy for ROCm 10.1 and gfx942:xnack+ that should move into workflow configuration;
  • the final fat wheel must be rebuilt after the compiler-rt and portable-RPATH dependencies land;
  • the independent kpack PEP 376 RECORD correction in ROCm/rocm-systems must land before the final split; and
  • strict offline GPU validation must replace the historical diagnostic baseline before publication.

Real-input evidence

The prototype consumed the Phase 1 10.1.0+asan.20260807 index and a HIPified PyTorch checkout based on dcf02e51371deb3050899643462349c1941e8979.

  • the full 3,830-edge Torch build completed and produced a 643 MiB fat wheel;
  • the isolated import sanity check reported 2.14.0a0+rocm10.1.asan.20260807;
  • corrected split host/device wheels passed ZIP CRC, full PEP 376 RECORD, reciprocal pin, kpack, and ASAN-linkage validation;
  • clean CPython 3.12 offline installation and plain Torch import passed;
  • the GPU smoke reached the known pre-fix compiler-rt null-PC initialization failure, so the preserved artifacts are diagnostic evidence, not publishable outputs.

No local runner, wheel, dependency wheelhouse, generated AOTriton images, PyTorch HIPIFY output, submodule pointer, or validation artifact is included.

Tests

20 focused ASAN production-builder tests passed
python -m py_compile: pass
git diff --check: pass

The three portable-RPATH tests from the integrated prototype were excluded because #7210 owns that implementation and its focused coverage.

Dependencies and final acceptance

After those dependencies land, rebuild rather than patch the preserved wheels and require zero absolute RPATH/RUNPATH components, strict offline installation, successful gfx942 tensor execution (1240.0), and zero downgraded ASAN findings.

@therock-pr-bot

therock-pr-bot Bot commented Aug 12, 2026

Copy link
Copy Markdown

❌ PR Check — Action Required

Check Status Details
📝 PR Description ❌ Fail Error: PR description must reference a JIRA ID, ISSUE ID, or a GitHub closing keyword.
Expected: include a JIRA ID / ISSUE ID line (separator : or -, or omitted; value may be a JIRA key, a number with/without #, or a link), OR a closing keyword + issue reference. Accepted examples:
JIRA ID : TESTAUTO-6039
JIRA ID - #330
JIRA ID #330
JIRA ID (on separate line)
ROCM-25757
ISSUE ID : TESTUTO-3334
ISSUE ID #3334
ISSUE ID - TESTAUTO-3433
ISSUE ID (on separate line)
AIRUNTIME-2352
ISSUE ID : https://github.com/<org_name>/<repo_name>/issues/1234
Closes #10
Fixes octo-org/octo-repo#100
Resolves: #123
#123
https://github.com/<org_name>/<repo_name>/issues/123
Current: no valid JIRA/ISSUE/closing-keyword reference found
Forbidden Files ✅ Pass
🧪 Unit Test ✅ Pass
🔎 pre-commit ❌ Fail Error: Check concluded with failure.
🚫 Draft PR 🔜 To Be Enabled
🚩 Feature Flag 🔜 To Be Enabled
📊 Code Coverage 🔜 To Be Enabled

⚠️ 2 policy check(s) failed. Please address the issues above before this PR can be Reviewed.

🚫 Please fix the failed policies

  • ❌ PR Description
  • ❌ pre-commit

The Not ready to Review label was added to this PR. Once all policies pass, the label is removed automatically.

📖 Need help? See the Policy FAQ for details on every check and how to fix failures.

🙋 Wish to Override Policy?

@therock-pr-bot therock-pr-bot Bot added the Not ready to Review PR has unresolved policy failures — reviews blocked label Aug 12, 2026
@therock-pr-bot

Copy link
Copy Markdown

🚫 Please fix the failed policies before requesting reviews.

The following policy checks failed:

  • ❌ PR Description

The Not ready to Review label has been added to this PR.
Once all policies pass, the label will be removed automatically.

@ScottTodd

Copy link
Copy Markdown
Member

Stack: 2 of 2, after #7312.

For the Phase 2-only review diff, compare draft/asan-wheels-phase1-rocm...draft/asan-wheels-phase2-pytorch.

You can stack your PR either with https://docs.github.com/en/pull-requests/how-tos/stacked-pull-requests or by changing the base branch for this PR to that other branch, fyi. Both require creating branches in the shared repository, which is what we recommend for AMD employees anyways: https://github.com/ROCm/TheRock/blob/main/CONTRIBUTING.md#branch-creation-policy.

ScottTodd pushed a commit that referenced this pull request Aug 14, 2026
)

## Motivation

Follow-up to #7294, where the request was for an off-the-shelf markdown
link
checker rather than fixing broken links one instance at a time.

## Technical Details

Three changes:

1. `.pre-commit-config.yaml` — the `lychee` block exactly as written in

[#7294](#7294 (comment)),
   plus `stages: [manual]`.
2. `external-builds/pytorch/README.md:199` — one leading `/`. The
sibling link
   two lines below at `:201` already has it.
3. `CONTRIBUTING.md` — the manual invocation, added to the existing
   `#### pre-commit checks` example block.

**No `exclude` is needed.** `.pre-commit-config.yaml:4` already carries
a global
`exclude: "third_party/"`, which covers `build_tools/third_party/`. That
is the
whole of the 3-vs-4 discrepancy: my 4 came from running the `lychee`
binary
directly over the tree, which applies no pre-commit file filtering; the
3 came
through `pre-commit`, which does. My earlier claim that the `implib`
link "needs
an exclude or the hook stays red" was wrong.

### Why `stages: [manual]`

Because two of the three links it reports cannot be fixed here — open
pull
requests rewrite both lines — so at the default stage the existing
`.github/workflows/pre-commit.yml` job would fail on every PR until they
land.

| Link | Open PR covering that line | Status |
| --- | --- | --- |
| `docs/development/README.md:24` | #7253, hunk `@@ -17,11 +17,11 @@`
deletes that exact line | left alone |
| `docs/development/workflow_outputs.md:305` | #4685, hunk `@@ -295,15
+295,15 @@` rewrites that exact line | left alone |
| `external-builds/pytorch/README.md:199` | none of the 8 open PRs on
the file; nearest hunks are #5439 `190-195` and #7313 `180-185` | fixed
here |

Hunk ranges are from each PR's combined diff against its own merge base.
The
scan covered all 481 open PRs.

### What `stages: [manual]` actually does

It skips the binary install, not the hook repo clone. Each row is a
fresh
`PRE_COMMIT_HOME`:

| Command | hook repo cloned | `lychee` binary installed | hook runs |
| --- | --- | --- | --- |
| `pre-commit install-hooks` | yes, 7.4 MB | no | no |
| `pre-commit run` | yes, 7.4 MB | no | no |
| `pre-commit run --all-files` | yes, 7.4 MB | no | no |
| `pre-commit run --hook-stage manual --all-files` | yes | yes, 31 MB
total, 109 s | yes |
| *without* `stages: [manual]`, `pre-commit run --all-files` | yes |
yes, 31 MB total, 121 s | yes |

So the `cargo` step you had to approve is skipped on commit and in CI;
the
7.4 MB `git clone` of `lycheeverse/lychee` still happens on any first
`pre-commit` invocation, including `install-hooks`.

Separately: the `lychee` hook id bootstrapped fine here on a clean cache
— it
fetches `cargo-binstall` and a prebuilt binary, no pre-existing `cargo`
needed.
My earlier report that it failed to bootstrap was a network restriction
on my
side, not the hook.

## Test Plan

`pre-commit` 4.6.2, Python 3.12.10, Windows 11, at `47947a1`. Each run
used an
isolated `PRE_COMMIT_HOME`. Linux and macOS were **not** exercised.

## Test Result

Default stage, whole config, whole tree — what CI runs. `lychee` is
absent from
the list, and the run is green:

```
$ pre-commit run --all-files
Trim Trailing Whitespace..................................................Passed
Fix End of Files..........................................................Passed
Check Yaml................................................................Passed
Check JSON................................................................Passed
Check for merge conflicts.................................................Passed
Check for added large files...............................................Passed
Mixed line ending.........................................................Passed
black.....................................................................Passed
clang-format..............................................................Passed
mdformat..................................................................Passed
No-tabs checker...........................................................Passed
Lint GitHub Actions workflow files........................................Passed
Forbid secrets, certificates, keys, and executables...(no files to check)Skipped
Enforce *_test.py naming for build_tools tests........(no files to check)Skipped
exit=0
```

Manual stage, after the `external-builds/pytorch/README.md` fix — 3
errors down
to 2, and the OK count moves 506 → 507:

```
$ pre-commit run --hook-stage manual --all-files lychee
Issues found in 2 inputs. Find details below.

[docs/development/README.md]:
[ERROR] .../docs/development/test_runner_info.md (at 24:3) | File not found.
[docs/development/workflow_outputs.md]:
[ERROR] .../build_tools/github_actions/upload_pytorch_manifest.py (at 305:3) | File not found.

🔍 2238 Total 🔗 1196 Unique ✅ 507 OK 🚫 2 Errors 👻 1729 Excluded
🔍 269 Total 🔗 241 Unique ✅ 55 OK 🚫 0 Errors 👻 214 Excluded
```

The `third_party` A/B, deleting only line 4 of `.pre-commit-config.yaml`
and
changing nothing else:

```
# line 4 present (as shipped) — 3 errors, implib is not even an input
🔍 2238 Total 🔗 1197 Unique ✅ 506 OK 🚫 3 Errors 👻 1729 Excluded

# line 4 deleted — the 4th error appears
[build_tools/third_party/implib/doc/ReduceLibraryInterface.md]:
[ERROR] .../build_tools/third_party/implib/doc/tests/multilib/run.sh (at 32:138) | File not found.
🔍 1177 Total 🔗 596 Unique ✅ 107 OK 🚫 1 Error 👻 1069 Excluded
```

If you want the class actually gated rather than merely available, the
existing
`pre-commit` job takes it in three lines — but only once the two links
above are
resolved, since it would be red until then:

```yaml
      - uses: pre-commit/action@2c7b380 # v3.0.1
        with:
          extra_args: --hook-stage manual --all-files lychee
```

Happy to send that as a follow-up after #7253 lands.

## Submission Checklist

- [x] Look over the contributing guidelines at
https://github.com/ROCm/TheRock/blob/main/CONTRIBUTING.md.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Not ready to Review PR has unresolved policy failures — reviews blocked

Projects

Status: TODO

Development

Successfully merging this pull request may close these issues.

2 participants