Skip to content

[compiler-rt][asan] Add Linux HSA/ROCm GPU memory support via HSA API… - #3856

Open
ampandey-AMD wants to merge 1 commit into
ROCm:amd-stagingfrom
ampandey-AMD:cleanup-pr
Open

[compiler-rt][asan] Add Linux HSA/ROCm GPU memory support via HSA API…#3856
ampandey-AMD wants to merge 1 commit into
ROCm:amd-stagingfrom
ampandey-AMD:cleanup-pr

Conversation

@ampandey-AMD

Copy link
Copy Markdown

… interceptors

Add AddressSanitizer support for ROCm/HSA GPU memory on Linux by intercepting host-side HSA APIs (init/shutdown, memory pool allocate/free, IPC attach/detach, vmem reserve/free, memory copies, and pointer queries).

Build / platform:

  • SANITIZER_AMDHSA is enabled by default on Linux (non-Android). The HSA path is inert at runtime until libhsa-runtime64.so is dlopen'd; no ROCm install is required to build compiler-rt.
  • Vendored HSA headers are shared with the AMDGPU offload plugin (offload/plugins-nextgen/amdgpu/dynamic_hsa/), avoiding drift and removing the old CMake ROCm HSA/COMgr header discovery.

Allocator architecture:

  • Keep CombinedAllocator host-only for all sanitizers.

  • Add DeviceCombinedAllocator as an ASan-only wrapper that composes the host allocator with AmdgpuDeviceAllocatorT. Host malloc/free still use the inner CombinedAllocator; Device allocations go through a separate AllocateDevice() path used by HSA interceptors.

  • ROCr entry points are resolved via dlopen/dlsym on the ROCr handle after hsa_init, avoiding RTLD_NEXT recursion.

  • GPU-only vmem reservations (without HSA_AMD_VMEM_ADDRESS_NO_REGISTER) are tracked separately via VmemGpuReserveTracker because the reserved VA is not host-writable for ASan chunk metadata.

  • Tests under test/asan/TestCases/AMDGPU/ gate on Linux + ROCm runtime detection; VMEM tests additionally require the hsa-vmem lit feature when HSA_VMEM_SUPPORTED is set.

… interceptors

Add AddressSanitizer support for ROCm/HSA GPU memory on Linux by intercepting
host-side HSA APIs (init/shutdown, memory pool allocate/free, IPC
attach/detach, vmem reserve/free, memory copies, and pointer queries).

Build / platform:

- `SANITIZER_AMDHSA` is enabled by default on Linux (non-Android). The HSA path
  is inert at runtime until `libhsa-runtime64.so` is dlopen'd; no ROCm install
  is required to build compiler-rt.
- Vendored HSA headers are shared with the AMDGPU offload plugin
  (`offload/plugins-nextgen/amdgpu/dynamic_hsa/`), avoiding drift and removing
  the old CMake ROCm HSA/COMgr header discovery.

Allocator architecture:

- Keep `CombinedAllocator` host-only for all sanitizers.
- Add `DeviceCombinedAllocator` as an ASan-only wrapper that composes the host
  allocator with `AmdgpuDeviceAllocatorT`. Host `malloc`/`free` still use the
  inner `CombinedAllocator`; Device allocations go through a separate
  `AllocateDevice()` path used by HSA interceptors.
- ROCr entry points are resolved via dlopen/dlsym on the ROCr handle after
  `hsa_init`, avoiding RTLD_NEXT recursion.
- GPU-only vmem reservations (without `HSA_AMD_VMEM_ADDRESS_NO_REGISTER`) are
  tracked separately via `VmemGpuReserveTracker` because the reserved VA is not
  host-writable for ASan chunk metadata.

- Tests under `test/asan/TestCases/AMDGPU/` gate on Linux + ROCm runtime
  detection; VMEM tests additionally require the `hsa-vmem` lit feature when
  `HSA_VMEM_SUPPORTED` is set.

Co-authored-by: Brian Sumner <Brian.Sumner@amd.com>
Co-authored-by: Bing Ma <Bing.Ma@amd.com>
@lamb-j

lamb-j commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator

Can this go upstream? Or can we get something in the description on why it has to be downstream only? Or is it just for testing?

@ampandey-AMD

Copy link
Copy Markdown
Author

Can this go upstream? Or can we get something in the description on why it has to be downstream only? Or is it just for testing?

There is already an upstreamed version of this patch , PR 192240. The downstream PR is there to avoid any merge conflicts with current downstream HEAD.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants