Fuse DeepSeek MTP verification into decode dispatch

[high-cloud]

Summary

• compile main decode plus device-side MTP verification as one L2 submission
• load the fused orchestration from the dedicated decode_fwd_mtp kernel module
• keep MTP draft decode as the second L2 submission within one serving dispatch
• preserve the separate autoregressive decode path when MTP is disabled
• teach the DSV4 profiling skill to detect current 2-L2 and legacy layouts
• update the pypto-lib submodule to merged origin/main commit fd91568

Why

DeepSeek V4 previously submitted main decode, MTP verification, and MTP draft decode as three L2 callables. The extra verification submission increased host dispatch overhead and heap-ring pressure. The final library layout keeps standalone JIT entrypoints independent from the inline fused implementation.

Impact

The fused path reduces decode L2 submissions from three to two. In the 8-card DP/EP=8 MTP profile with 20 output tokens on a2a3 and PTOAS 0.54, steady critical-rank host decode time decreased from 45.159 ms to 42.980 ms. The same workload completed with a 2 GiB ring heap.

Validation

• 41 DeepSeek V4 unit tests passed
• Ruff and diff checks passed
• standalone lib decode_fwd and decode_mtp passed on real a2a3 devices
• 8-card DeepSeek V4 HTTP accuracy guard passed: 4 tests in 624.85 s
• server compiled _jit_l3_decode_fwd_mtp and returned the expected 10-token completion

Uses merged hw-native-sys/pypto-lib#884.

[coderabbitai]

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 991b5232-001f-4cf8-a2bc-ec6000d2ccc3

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

📝 Walkthrough

Walkthrough

Changes

DeepSeek V4 fused MTP decode

Layer / File(s)	Summary
Fused decode compilation pypto-lib, pypto_serving/model/deepseek/npu_executor.py	The executor builds one fused main/MTP decode callable and conditionally loads MTP kernels.
Fused runner dispatch pypto_serving/model/deepseek/npu_runner.py	The runner stages tail metadata, performs one fused dispatch, reads device outputs, and updates request state.
Fused decode validation tests/test_deepseek_v4.py	Tests cover disabled MTP, shared-worker configuration, fused dispatch, accepted tokens, and draft state updates.
Decode layout profiling .agents/skills/profile-dsv4-serving-strace/*	Profiling detects fused two-invocation and split four-invocation layouts and reports layout-specific metrics.
Estimated code review effort: 4 (Complex)	~60 minutes

Sequence Diagram(s)

sequenceDiagram
  participant DeepSeekV4ModelRunner
  participant fused_decode_callable
  participant device_buffers
  DeepSeekV4ModelRunner->>device_buffers: stage tail tokens, positions, and slots
  DeepSeekV4ModelRunner->>fused_decode_callable: dispatch combined main and MTP arguments
  fused_decode_callable->>device_buffers: write acceptance counts and sampled IDs
  DeepSeekV4ModelRunner->>device_buffers: read outputs and update request state

Loading

Possibly related PRs

• hw-native-sys/pypto-serving#89: Earlier DeepSeek V4 MTP implementation that this change replaces with fused dispatch.
• hw-native-sys/pypto-serving#117: Related MTP device execution and shared-buffer state changes.
• hw-native-sys/pypto-serving#125: Related profiling support for fused MTP decode layouts.

Poem

A rabbit watched one decode hop,
Main and MTP no longer stop.
Tail tokens travel, buffers glow,
Accepted drafts return in flow.
Two lanes or four, the traces show—
Fused hops help the metrics grow.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly summarizes the main change: fusing DeepSeek MTP verification into the decode dispatch.
Description check	✅ Passed	The description accurately covers the fused decode implementation, profiling updates, submodule change, performance impact, and validation.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🧹 Nitpick comments (1)

tests/test_deepseek_v4.py (1)

1354-1449: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Add direct unit coverage for _stage_fused_mtp_metadata.

This test exercises the fused decode dispatch end-to-end, but it stubs _stage_fused_mtp_metadata (runner._stage_fused_mtp_metadata = lambda _inputs: layout.decode_seq). Its staging logic — writing decode_tail_token_ids/decode_tail_positions/decode_tail_slot_ids, computing decode_logit_row_indices, and raising RuntimeError when committed tail state is missing — is not directly verified anywhere in this diff.

Add a focused unit test for _stage_fused_mtp_metadata, similar to test_deepseek_stage_mtp_decode_inputs_updates_only_active_prefix_after_first_step for the legacy _stage_mtp_decode_inputs.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/test_deepseek_v4.py` around lines 1354 - 1449, Add a focused unit test
for runner._stage_fused_mtp_metadata, modeled on
test_deepseek_stage_mtp_decode_inputs_updates_only_active_prefix_after_first_step.
Verify it writes decode_tail_token_ids, decode_tail_positions, and
decode_tail_slot_ids, computes decode_logit_row_indices correctly, and raises
RuntimeError when committed tail state is missing; keep the existing
fused-dispatch test unchanged.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.agents/skills/profile-dsv4-serving-strace/scripts/analyze_profile.py:
- Around line 339-361: Update the device-effective decode report and table
generation to use the existing main_phase_label variable from the decode-layout
branch instead of hardcoded “main” or “Main Effective” labels. This ensures
two_l2 traces with device STRACE display “Main+verify” consistently while
preserving “Main” for other layouts.

---

Nitpick comments:
In `@tests/test_deepseek_v4.py`:
- Around line 1354-1449: Add a focused unit test for
runner._stage_fused_mtp_metadata, modeled on
test_deepseek_stage_mtp_decode_inputs_updates_only_active_prefix_after_first_step.
Verify it writes decode_tail_token_ids, decode_tail_positions, and
decode_tail_slot_ids, computes decode_logit_row_indices correctly, and raises
RuntimeError when committed tail state is missing; keep the existing
fused-dispatch test unchanged.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: e1118d41-ae95-4b06-a77e-9bde172ac3e4

📥 Commits

Reviewing files that changed from the base of the PR and between e153b81 and 0a24f3a.

📒 Files selected for processing (7)

• .agents/skills/profile-dsv4-serving-strace/SKILL.md
• .agents/skills/profile-dsv4-serving-strace/scripts/analyze_profile.py
• .agents/skills/profile-dsv4-serving-strace/scripts/render_8lane.py
• pypto-lib
• pypto_serving/model/deepseek/npu_executor.py
• pypto_serving/model/deepseek/npu_runner.py
• tests/test_deepseek_v4.py