HAMi-DRA

A Kubernetes mutating webhook that converts GPU device resources to Dynamic Resource Allocation (DRA) ResourceClaims, with an optional metrics monitor and a fake DRA driver for testing.

Overview

This webhook automatically transforms Pod specifications that request GPU resources (e.g., nvidia.com/gpu) into DRA ResourceClaims, enabling dynamic resource allocation for GPU workloads in Kubernetes.

Features

• Automatic Resource Conversion: Converts GPU resource requests to ResourceClaims
• Resource Cleanup: Automatically removes GPU resources from Pod specs and creates corresponding ResourceClaims
• Annotation Support: Supports device selection via Pod annotations (UUID, device type)
• Multi-vendor: NVIDIA, Hygon HCU, AMD and Ascend NPU in one deployment
• Metrics Monitoring: Optional monitor component that collects and exposes GPU resource metrics via Prometheus
• Fake Driver: A fake DRA driver for testing without GPU hardware, see docs/fake-dra-driver.md

Installation

Prerequisites

• Kubernetes version >= 1.34 with the DRAConsumableCapacity featuregate enabled (alpha in 1.34, beta and on by default since 1.36)
• CDI must be enabled in the underlying container runtime (such as containerd or CRI-O).
• NVIDIA GPU Driver 440 or later

Configure and install with Helm

You need cert-manager installed before installing the webhook. If you don't want to use cert-manager, set certs.certManager.enabled=false and provide your own certificate via certs.custom.crt and certs.custom.key, and the CA that signed it (base64 encoded PEM) via certs.custom.caBundle unless the API server already trusts it.

Add the HAMi-DRA Helm repository:

helm repo add hami-dra https://project-hami.github.io/HAMi-DRA
helm repo update

Install the chart:

helm install hami-dra hami-dra/hami-dra

To upgrade to the latest version in the future:

helm repo update
helm upgrade hami-dra hami-dra/hami-dra

If you are not using gpu-operator provided containerd drivers, you can use the following command to install the webhook:

helm install hami-dra hami-dra/hami-dra \
  --set drivers.nvidia.containerDriver=false

Then use the same as hami.

Hygon HCU

For clusters running Hygon HCU with k8s-hcu-dra-driver, see docs/hygon-hcu.md.

Configuration

Device Resources

Configure device resources via --set flags or a custom values.yaml. The default resource names are:

resourceName: "nvidia.com/gpu"
resourceMem: "nvidia.com/gpumem"
resourceCores: "nvidia.com/gpucores"

The webhook can convert multiple device vendors in one deployment. Configure deviceVendors, for example:

deviceVendors:
  - nvidia
  - hygon
  - ascend
  - amd

The former single-value deviceVendor setting has been removed. See Hygon HCU and Ascend DRA. For AMD, add amd to deviceVendors, enable drivers.amd, and set amdReferenceComputeUnits (see the comments in charts/hami-dra/values.yaml). The generated webhook configuration uses the equivalent vendors list. Direct webhook deployments can override it with --device-vendors=nvidia,hygon,ascend,amd.

ResourceClaimTemplate

By default the webhook creates a ResourceClaim and points the Pod at it. Set the following to create a ResourceClaimTemplate instead, for every converted Pod. Kubernetes generates the ResourceClaim from that template:

webhook:
  config:
    resourceClaimTemplate: true

Kueue can admit this form. It rejects Pods that reference a ResourceClaim directly. Kueue must map the HAMi DeviceClass in its deviceClassMappings configuration, otherwise the Workload stays inadmissible. Workloads for which Kueue builds the Workload from the parent object before the Pod exists, such as a batch/v1 Job, are not covered yet.

Monitor Component

The monitor component is an optional feature that collects and exposes GPU resource metrics via Prometheus. It is enabled by default.

Quick Start:

Set the monitor service to NodePort so we can access it outside the cluster:

monitor:
  enabled: true
  service:
    type: NodePort
    nodePort:
      metrics: 31995

Access metrics:

# With NodePort
curl http://<node-ip>:31995/metrics

You will see metrics like this:

For detailed configuration, metrics documentation, and Prometheus integration, see MONITOR.md.