Releases: amd/MxGPU-Virtualization
GIM 8.6.0.K Release Note
We are excited to announce a new open-source release of our GIM SR-IOV Driver.
This release has regular bug fixes and improvements.
Hardware / Features Supported for this Release
| Hardware | Host OS | Guest OS | ROCm version | Number of VFs per GPU | Infinity Fabric (XGMI) Configuration |
|---|---|---|---|---|---|
| AMD Instinct MI355X | Ubuntu 24.04 | RHEL 9.6, RHEL 10, Ubuntu 24.04 | ROCm 7.1.1 | 1 | between 8 GPUs |
| AMD Instinct MI350X | Ubuntu 24.04 | RHEL 9.6, RHEL 10, Ubuntu 24.04 | ROCm 7.1.1 | 1 | between 8 GPUs |
| AMD Instinct MI325X | Ubuntu 22.04 | Ubuntu 22.04 | ROCm 7.1.1 | 1 | between 8 GPUs |
| AMD Instinct MI300X | Ubuntu 22.04 | Ubuntu 22.04 | ROCm 7.1.1* | 1 or 8* | between 8 GPUs |
| AMD Instinct MI300X | RHEL 9.4 | RHEL 9.4, Ubuntu 24.04 | ROCm 7.1.1* | 1 or 8* | between 8 GPUs |
| AMD Instinct MI210X | RHEL 9.4 | RHEL 9.4, Ubuntu 22.04 | ROCm 7.1.1 | 1 | between 4/8 GPUs |
| AMD Radeon PRO V710 | Ubuntu 24.04 | Ubuntu 24.04 | ROCm 7.1.1 | 1 or 4 or 12 | N/A |
Partition Mode Support
| Number of VFs per GPU | Dynamic Compute Partitioning | NPS1 | NPS2 | NPS4 |
|---|---|---|---|---|
| 1 | SPX (Default) | MI300X, MI325X, MI35XX | ||
| 1 | DPX | MI35XX | ||
| 1 | CPX | MI35XX (Preview) | ||
| 8* | CPX | MI300X (Preview) |
For the full list of supported partition modes and the relationship between partition mode and number of VFs per GPU, see the GPU Partitioning.
* Multi-VF (8 VF) Support requires a compatible firmware BKC bundle for the GPU.
Please refer to the ROCm Firmware Compatibility Table for the list of required firmware versions and supported configurations.
Host AMD SMI Versions
- Library: 50.2.2
- Tool: 32.4.1
Please checkout out SMI doc for more details.
Getting Started
Please check out our User Guide for instructions on how to set up GIM and example configurations to run SR-IOV enabled VMs.
Known Issues
- When using the configuration of 64 ROCM devices in 1 VM (aka CPX mode with 1 VF) on a MI350 platform, we would see a hang in workloads which use SDMA engines
- MultiVF configuration for MI300X requires specific firmware version. please refer to the ROCm compatibility matrix for supported version
- MI210: Unloading the Guest driver on MI210 may produce kernel call trace. Note that this is just a warning and not a kernel panic. Guest VM reboot is required to restore the operation β
- SMI CLI topology --link_type command returns wrong value for the first BDFβ
- SMI CLI static --vram command returns 0 for the bit width valueβ
- Some BDF values returned from SMI get_link_topology_nearest APIs are wrong
GIM 8.5.0.K Release Note
We are excited to announce a new open-source release of our GIM SR-IOV Driver.
This release adds MI350 CPX support and SMI CLI command improvement on top of the regular update and bug fixes.
Hardware / Features Supported for this Release
| Hardware | Host OS | Guest OS | ROCm version | Number of VFs per GPU | Infinity Fabric (XGMI) Configuration |
|---|---|---|---|---|---|
| AMD Instinct MI355X | Ubuntu 24.04 | RHEL 9.6, RHEL 10, Ubuntu 24.04 | ROCm 7.1 | 1 | between 8 GPUs |
| AMD Instinct MI350X | Ubuntu 24.04 | RHEL 9.6, RHEL 10, Ubuntu 24.04 | ROCm 7.1 | 1 | between 8 GPUs |
| AMD Instinct MI325X | Ubuntu 22.04 | Ubuntu 22.04 | ROCm 7.1 | 1 | between 8 GPUs |
| AMD Instinct MI300X | Ubuntu 22.04 | Ubuntu 22.04 | ROCm 7.1 | 1 or 8* | between 8 GPUs |
| AMD Instinct MI300X | RHEL 9.4 | RHEL 9.4 | ROCm 7.1 | 1 or 8* | between 8 GPUs |
| AMD Instinct MI210X | RHEL 9.4 | RHEL 9.4, Ubuntu 22.04 | ROCm 7.1 | 1 | between 4/8 GPUs |
| AMD Radeon PRO V710 | Ubuntu 24.04 | Ubuntu 24.04 | ROCm 7.1 | 1 or 4 or 12 | N/A |
Partition Mode Support
| Number of VFs per GPU | Dynamic Compute Partitioning | NPS1 | NPS2 | NPS4 |
|---|---|---|---|---|
| 1 | SPX (Default) | MI300X, MI325X, MI35XX | ||
| 1 | DPX | MI35XX | ||
| 1 | CPX | MI35XX (Preview) | ||
| 8* | CPX | MI300X (Preview) |
For the full list of supported partition modes and the relationship between partition mode and number of VFs per GPU, see the GPU Partitioning.
* Multi-VF (8 VF) Support requires a compatible firmware BKC bundle for the GPU.
Please refer to the ROCm Firmware Compatibility Table for the list of required firmware versions and supported configurations.
Host AMD SMI Versions
- Library: 40.0.6
- Tool: 30.0.3
Please checkout out SMI doc for more details.
Getting Started
Please check out our User Guide for instructions on how to set up GIM and example configurations to run SR-IOV enabled VMs.
Known Issues
With NPS1/SPX and 8VMs (1VF per VM) on MI300, triggering reset via "/sys/kernel/debug/kfd/hang_hws" causes the GPU to become unusable. Rebooting the guest should solve the problem.
GIM 8.4.1.K Release Note
We are excited to announce a new open-source release of our GIM SR-IOV Driver.
This release adds VF FLR improvement on Navi32 as well as MI300 multi Vf on top of the regular update and bug fixes.
Hardware / Features Supported for this Release
| Hardware | Host OS | Guest OS | ROCm version | Number of VFs per GPU | Infinity Fabric (XGMI) Configuration |
|---|---|---|---|---|---|
| AMD Instinct MI355X | Ubuntu 24.04 | Ubuntu 24.04 | ROCm 7.0.2 | 1 | between 8 GPUs |
| AMD Instinct MI350X | Ubuntu 24.04 | Ubuntu 24.04 | ROCm 7.0.2 | 1 | between 8 GPUs |
| AMD Instinct MI325X | Ubuntu 22.04 | Ubuntu 22.04 | ROCm 7.0.2 | 1 | between 8 GPUs |
| AMD Instinct MI300X | Ubuntu 22.04 | Ubuntu 22.04 | ROCm 7.0.2 | 1 or 8* | between 8 GPUs |
| AMD Instinct MI300X | RHEL 9.4 | RHEL 9.4 | ROCm 7.0.2 | 1 or 8* | between 8 GPUs |
| AMD Instinct MI210X | RHEL 9.4 | RHEL 9.4, Ubuntu 22.04 | ROCm 7.0.2 | 1 | between 4/8 GPUs |
| AMD Radeon PRO V710 | Ubuntu 24.04 | Ubuntu 24.04 | ROCm 7.0.2 | 1 or 4 or 12 | N/A |
Partition Mode Support
| Number of VFs per GPU | Dynamic Compute Partitioning | NPS1 | NPS2 | NPS4 |
|---|---|---|---|---|
| 1 | SPX (Default) | MI300X, MI325X, MI35XX | ||
| 1 | DPX | MI35XX (preview) | ||
| 1 | CPX | |||
| 8* | CPX | MI300X (preview) |
For the full list of supported partition modes and the relationship between partition mode and number of VFs per GPU, see the GPU Partitioning.
* Multi-VF (8 VF) Support requires a compatible firmware BKC bundle for the GPU.
Please refer to the ROCm Firmware Compatibility Table for the list of required firmware versions and supported configurations.
Host AMD SMI Versions
- Library: 33.1.3
- Tool: 27.8.0
Since the last release, the library and tool will be automatically installed when installing the gim-dkms packageβmanual build step is no longer required.
If you manually copied AMD SMI tool/library to /usr, you may need to reset the ldconfig cache and/or remove the AMD SMI files under /usr.
Getting Started
Please check out our User Guide for instructions on how to set up GIM and example configurations to run SR-IOV enabled VMs.
Known Issues
- Ubuntu kernel
6.8.0-64and6.14.0-24and possibly other kernels on other distros have a regression.
Please update to a later kernel that contains the fix. - For V710, the VF active time reported by
amd-smiis not accurate.
GIM 8.4.0.K Release Note
We are excited to announce a new open-source release of our GIM SR-IOV Driver.
This release adds dynamic partitioning support as well as V710 validation on top of the regular update and bug fixes.
Hardware / Features Supported for this Release
| Hardware | Host OS | Guest OS | ROCm version | Number of VFs per GPU | Infinity Fabric (XGMI) Configuration |
|---|---|---|---|---|---|
| AMD Instinct MI355X | Ubuntu 24.04 | Ubuntu 24.04 | ROCm 7.0 | 1 | between 2/4/8 GPUs |
| AMD Instinct MI350X | Ubuntu 24.04 | Ubuntu 24.04 | ROCm 7.0 | 1 | between 2/4/8 GPUs |
| AMD Instinct MI325X | Ubuntu 22.04 | Ubuntu 22.04 | ROCm 7.0 | 1 | between 8 GPUs |
| AMD Instinct MI300X | Ubuntu 22.04 | Ubuntu 22.04 | ROCm 7.0 | 1 | between 2/4/8 GPUs |
| AMD Instinct MI300X | RHEL 9.4 | RHEL 9.4 | ROCm 7.0 | 1 | between 2/4/8 GPUs |
| AMD Instinct MI210X | RHEL 9.4 | RHEL 9.5, Ubuntu 22.04 | ROCm 7.0 | 1 | between 4/8 GPUs |
| AMD Radeon PRO V710 | Ubuntu 24.04 | Ubuntu 24.04 | ROCm 7.0 | 1 / 4 / 12 | N/A |
Partition Mode Support
| Partition Mode | SPX | DPX | QPX | CPX |
|---|---|---|---|---|
| NPS1 | MI35XX, MI325X, MI300X | |||
| NPS2 | MI35XX (preview) | |||
| NPS4 | MI325X, MI300X |
For the full list of supported partition modes and the relationship between partition mode and number of VFs per GPU, see the GPU Partitioning.
Host AMD SMI Versions
- Library: 33.1.3
- Tool: 27.8.0
Since the last release, the library and tool will be automatically installed when installing the gim-dkms packageβmanual build step is no longer required.
If you manually copied AMD SMI tool/library to /usr, you may need to reset the ldconfig cache and/or remove the AMD SMI files under /usr.
Getting Started
Please check out our User Guide for instructions on how to set up GIM and example configurations to run SR-IOV enabled VMs.
Known Issues
- Ubuntu kernel
6.8.0-64and6.14.0-24and possibly other kernels on other distros have a regression.
Please update to a later kernel that contains the fix. - For V710, the VF active time reported by
amd-smiis not accurate.
GIM 8.3.0.K Release Note
We are excited to announce a new open-source release of our GIM SR-IOV Driver. This release adds MI325 validation on top of the regular update and bug fixes.
Hardware/Features tested for this release:
| Hardware | Host OS | Guest OS | ROCm version | Number of VFs per GPU | Infinity Fabric (XGMI) Configuration |
|---|---|---|---|---|---|
| AMD Instinct MI350X | Ubuntu 24.04 | Ubuntu 24.04 | To be announced | 1 | 1/2/4/8 VF in a single VM |
| AMD Instinct MI325X | Ubuntu 22.04 | Ubuntu 22.04 | ROCm 6.4 | 1 | 1/8 VF in a single VM |
| AMD Instinct MI300X | Ubuntu 22.04 | Ubuntu 22.04 | ROCm 6.4 | 1 | 1/2/4/8 VF in a single VM |
| AMD Instinct MI300X | RHEL 9.4 | RHEL 9.4 | ROCm 6.4 | 1 | 1/2/4/8 VF in a single VM |
| AMD Instinct MI210X | RHEL 9.4 | RHEL 9.5, Ubuntu 24.04 | ROCm 6.4 | 1 | 1/4/8 VF in a single VM |
Host AMD SMI Versions:
Library: 30.0.1
Tool: 27.4.0
Starting with this release, the library and tool will be automatically installed when installing gim-dkms package, manual build step is no longer required. If you had manually copied amd smi tool/library to /usr, you may need to reset ldconfig cache and/or remove the amd smi files under /usr.
Getting Started:
Please check out our User Guide for instructions on how to set up GIM and example configurations to run SR-IOV enabled VMs.
Known issues:
The -follow flag for amd-smi ras command is not working, to query CPERs run the command without the --follow flag.
GIM 8.2.0.K Release Note
We are excited to announce a new open-source release of our GIM SR-IOV Driver. Besides regular update and bug fixes, this release also contains development preview for MI350X.
Hardware/Features tested for this release:
| Hardware | Host OS | Guest OS | ROCm version | Number of VFs per GPU | Infinity Fabric (XGMI) Configuration |
|---|---|---|---|---|---|
| AMD Instinct MI350X | Ubuntu 24.04 | Ubuntu 24.04 | To be announced | 1 | 8 VF in a single VM |
| AMD Instinct MI300X | Ubuntu 22.04 | Ubuntu 22.04 | ROCm 6.4 | 1 | 1/2/4/8 VF in a single VM |
| AMD Instinct MI300X | RHEL 9.4 | RHEL 9.4 | ROCm 6.4 | 1 | 8 VF in a single VM |
| AMD Instinct MI210X | RHEL 9.4 | RHEL 9.5, Ubuntu 24.04 | ROCm 6.4 | 1 | 1/4/8 VF in a single VM |
Host AMD SMI Versions:
Library: 28.5.0
Tool: 25.5.0
Getting Started:
Please check out our User Guide for instructions on how to set up GIM and example configurations to run SR-IOV enabled VMs.
Known issues:
amdsmi_clear_vf_fb does not clear vf fb buffer.
GIM 8.1.0.K Release Note
We are excited to announce a new open-source release of our GIM SR-IOV Driver. The SMI library version for this release is 28.3.2, and the smi tool version is 25.3.1
Hardware/Features supported by this release:
| Hardware | Supported Host OS | Supported Guest OS/ROCm version | Number of VFs per GPU | Infinity Fabric (XGMI) Configuration |
|---|---|---|---|---|
| AMD Instinct MI300X | Ubuntu 22.04 | Ubuntu 22.04/ROCm 6.4 | 1 | 1/2/4/8 VF in a single VM |
| AMD Instinct MI300X | RHEL 9.4 | RHEL 9.4/ROCm 6.4 | 1 | 8 VF in a single VM |
| AMD Instinct MI210X | RHEL 9.4 | RHEL 9.5, Ubuntu 24.04/ROCm 6.4 | 1 | 1/4/8 VF in a single VM |
Getting Started:
Please check out our User Guide for instructions on how to set up GIM and example configurations to run SR-IOV enabled VMs.
Known issues:
SMI API amdsmi_clear_vf_fb does not clear vf fb buffer.
Driver reload could trigger a rare race condition that causes SMI to show wrong XGMI topology.
GIM 8.0.0.K Release Note
We are excited to announce a new open-source release of our GIM SR-IOV Driver.
Hardware/Features supported by this release:
| Hardware | Supported Host OS | Supported Guest OS/ROCm version | Number of VFs per GPU |
|---|---|---|---|
| AMD Instinct MI300X | Ubuntu 22.04 | Ubuntu 22.04/ROCm 6.4 | 1 |
Getting Started:
Please check out our User Guide for instructions on how to set up GIM and example configurations to run SR-IOV enabled VMs.
Known issues:
SMI API amdsmi_clear_vf_fb does not clear vf fb buffer