Skip to content

[WebGPU] fixes pointer arithmetic bug for multi-image inputs for mistral3 - #2510

Merged
kunal-vaishnavi merged 3 commits into
mainfrom
prathikrao/webgpu-multi-image-input-bugfix
Sep 1, 2026
Merged

kunal-vaishnavi merged 3 commits into
mainfrom
prathikrao/webgpu-multi-image-input-bugfix

Conversation

@prathikr

Copy link
Copy Markdown
Contributor

Fix WebGPU multi-image Pixtral output handling

The customer's native multi-image repro shows that two-image Ministral/Pixtral inference succeeds on CPU and that each image succeeds independently on WebGPU, but the two-image WebGPU case fails in Generator.SetInputs, including repeatable 0xC0000005 native access violations.

PixtralVisionState ran vision once per image and wrote each result into an offset view of the combined image_features output. The WebGPU comment in src/ep/webgpu/interface.cpp identified the issue: p_device_ is a WGPUBuffer handle cast to uint8_t*, not byte-addressable memory; p_device_ + offset is therefore not a valid sub-buffer handle. The first image uses offset zero, while the second image does not.

This change runs each image into a standalone device output tensor and copies it into the combined feature tensor through ByteWrapTensor(...).subspan(...).CopyFrom(...). That path handles nonzero WebGPU offsets safely instead of constructing invalid handle-derived tensor views.

@prathikr
Prathik Rao (prathikr) requested a review from a team as a code owner August 31, 2026 21:04
Copilot AI lite review requested due to automatic review settings August 31, 2026 21:04

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes multi-image Pixtral/Ministral vision-output handling by avoiding invalid pointer arithmetic on non-byte-addressable device buffer handles (notably WebGPU), preventing Generator.SetInputs access violations when num_images_ > 1.

Changes:

  • Stops constructing per-image output tensor views via feat_raw + offset, which is unsafe for EPs where p_device_ is an opaque handle (e.g., WebGPU).
  • Runs the vision encoder into a standalone per-image output tensor and copies results into the combined image_features buffer using ByteWrapTensor(...).subspan(...).CopyFrom(...).

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/models/multi_modal.cpp
Comment thread test/python/models/test_mistral3_tokens.py Fixed
…tatement in function'

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
@kunal-vaishnavi
kunal-vaishnavi enabled auto-merge (squash) September 1, 2026 20:07
@kunal-vaishnavi
kunal-vaishnavi merged commit f31a3ef into main Sep 1, 2026
80 of 88 checks passed
@kunal-vaishnavi
kunal-vaishnavi deleted the prathikrao/webgpu-multi-image-input-bugfix branch September 1, 2026 20:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants