Skip to content

[FEA]: Support CU_LAUNCH_ATTRIBUTE_ACCESS_POLICY_WINDOW in LaunchConfig #2626

Description

@lijinf2

Is this a duplicate?

Area

cuda.core

Is your feature request related to a problem? Please describe.

LaunchConfig does not expose CU_LAUNCH_ATTRIBUTE_ACCESS_POLICY_WINDOW, which sets an L2 access-policy window for a launch (Ampere+).

Parent tracking: #496.

Describe the solution you'd like

  • Users can set CU_LAUNCH_ATTRIBUTE_ACCESS_POLICY_WINDOW via LaunchConfig.
  • Mapping to the native launch attribute is tested.

Describe alternatives you've considered

Use cuda.bindings.driver (CUlaunchAttribute / CU_LAUNCH_ATTRIBUTE_ACCESS_POLICY_WINDOW) and cuLaunchKernelEx directly.

Downside: leaves cuda.core users without a first-class LaunchConfig API; they must drop to low-level bindings, manage the attribute array themselves, and lose consistency with other LaunchConfig launch attributes (is_cooperative, programmatic_stream_serialization, etc.).

Additional context

Incremental LaunchConfig attribute coverage under #496 (same approach as #1334 for programmatic_stream_serialization).

Example (C API) — agent-generated; reviewed but not executed:

// 1. Reserve L2 cache space
cudaDeviceProp prop;
cudaGetDeviceProperties(&prop, 0);
size_t l2_setaside = min((size_t)(prop.l2CacheSize * 0.75),
                         (size_t)prop.persistingL2CacheMaxSize);
cudaDeviceSetLimit(cudaLimitPersistingL2CacheSize, l2_setaside);

// 2. Configure access policy window
size_t window_size = min((size_t)prop.accessPolicyMaxWindowSize,
                         num_bytes);

CUlaunchAttribute attr;
attr.id = CU_LAUNCH_ATTRIBUTE_ACCESS_POLICY_WINDOW;
attr.value.accessPolicyWindow.base_ptr = (void*)data_persistent;
attr.value.accessPolicyWindow.num_bytes = window_size;
attr.value.accessPolicyWindow.hitRatio  = 0.6;
attr.value.accessPolicyWindow.hitProp   = CU_ACCESS_PROPERTY_PERSISTING;
attr.value.accessPolicyWindow.missProp  = CU_ACCESS_PROPERTY_STREAMING;

// 3. Launch with cuLaunchKernelEx
CUlaunchConfig config = {
    .gridDimX = grid, .gridDimY = 1, .gridDimZ = 1,
    .blockDimX = 256, .blockDimY = 1, .blockDimZ = 1,
    .sharedMemBytes = 0,
    .hStream = stream,
    .attrs = &attr,
    .numAttrs = 1
};

for (int i = 0; i < 10; i++) {
    cuLaunchKernelEx(&config, kernelA, NULL, NULL);
}

// 4. Disable window (release L2 for other kernels)
attr.value.accessPolicyWindow.num_bytes = 0;
cuLaunchKernelEx(&config, kernelB, NULL, NULL);

// 5. Clear persisting L2 lines
cuCtxResetPersistingL2Cache();

Metadata

Metadata

Assignees

Labels

cuda.coreEverything related to the cuda.core modulefeatureNew feature or request

Type

No type

Projects

  • Status
    Todo

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions