[Bug] qwen3.5-2b-text-cuda-gpu crashes on first message: CUDA failure 1 (invalid argument) in LinearAttention node

[jkher]

Describe the issue

Running qwen3.5-2b-text-cuda-gpu:1 via foundry run and sending any chat message immediately throws an unhandled OnnxRuntimeGenAIException from a custom LinearAttention CUDA kernel. The model downloads and loads successfully, but generation fails on the very first prompt.

Error: error: IPC error 'op_handler_failed': Error from chat_completions command: Error: Microsoft.ML.OnnxRuntimeGenAI.OnnxRuntimeGenAIException: Non-zero status code returned while running
LinearAttention node. Name:'/model/layers.0/linear_attn/LinearAttention' Status Message: CUDA failure 1: invalid argument ; GPU=0 ; hostname=LAPTOP ;
file=C:\a_work\1\s\onnxruntime\contrib_ops\cuda\bert\linear_attention_impl.cu ; line=702 ; expr=attr_err;
at Microsoft.ML.OnnxRuntimeGenAI.Result.VerifySuccess(IntPtr) + 0x47
at Microsoft.ML.OnnxRuntimeGenAI.Generator.AppendTokenSequences(Sequences) + 0x1f
at Microsoft.Neutron.OpenAI.Provider.OnnxChatGenerator..ctor(OnnxLoadedModel, GeneratorParams, ILogger, Sequences, NamedTensors) + 0x94
at Microsoft.Neutron.OpenAI.Provider.OnnxChatGenerator.CreateOnnxChatGenerator(ChatCompletionCreateRequestExtended, OnnxLoadedModel, AzureFoundryLocalModel, ITelemetry,
ILogger) + 0xa94
at Microsoft.AI.Foundry.Local.ChatClient.<>c__DisplayClass8_0.b__0(CancellationToken) + 0x2a
at Microsoft.Neutron.OpenAI.Provider.ChatCompletions.d__3.MoveNext() + 0x234
--- End of stack trace from previous location ---
at Microsoft.AI.Foundry.Local.ChatClient.d__8.MoveNext() + 0x2cb
--- End of stack trace from previous location ---
at Microsoft.AI.Foundry.Local.ChatClient.d__8.MoveNext() + 0x446
--- End of stack trace from previous location ---
at Microsoft.AI.Foundry.Local.NativeInterop.<>c__DisplayClass13_0.<b__2>d.MoveNext() + 0x467
--- End of stack trace from previous location ---
at Microsoft.AI.Foundry.Local.NativeInterop.<>c__DisplayClass13_0.<b__2>d.MoveNext() + 0x7d9
--- End of stack trace from previous location ---
at Microsoft.AI.Foundry.Local.NativeInterop.d__9.MoveNext() + 0xb8

To reproduce

1. foundry run qwen3.5-2b-text (auto-selects qwen3.5-2b-text-cuda-gpu:1)
2. Wait for model download/load to complete
3. Type any message, e.g. Hi, and press enter

Urgency

No response

Platform and architecture

Windows X64

OS Version

10.0.26200 N/A Build 26200

Installation type

Released package/binary

Foundry Local version

0.10.3

API or surface area

CLI

Hardware acceleration/backend

WebGPU/CUDA

Backend/runtime version

CUDA release 12.8, V12.8.93