Repository navigation
Conversation
Independent Kimi XT Review — native frozen MTPThe original report reviewed base Real CLI model: Current delivery chooses native #2292 only. The older reports describe #2289 as an alternative because that was the review-time context; it is no longer part of the selected delivery. Current recipe and combined-test status is summarized in the updated PR body and developer guide. Review — XTuner #2292 Native frozen MTP + fused MTP expert loader
Summary本 PR 做两件事:
总体评价:改动小而收敛,抽象合理,无确认的功能缺陷。发现的问题集中在文档挂接(toctree 缺失)和少量设计/测试建议,见各章节 X2292-F01~F08。关键链路(frozen 参数 → 排除优化器 → 保留发布 → 保留 load spec → DCP 契约)经源码逐环核对一致。 Main Flowchart before this PRgraph TD
subgraph MTPConfig["xtuner.v1.module.mtp.config.MTPConfig"]
C1[num_layers / share_weights /<br/>loss_type / detach_* ]
end
subgraph MoE["xtuner.v1.model.moe.moe.MoE"]
M1["__init__: build_mtp_block()<br/>(mtp_config 非空即建)"]
M2["update_bias(): 主层 router + MTP gate<br/>行数 = num_hidden_layers + mtp 物理层"]
M3["build_loss_ctx_batch(): mtp_config 非空<br/>即建 mtp / mtp_e2e_tv ctx"]
M4["_forward(): mtp_block 非空即跑<br/>MTP training forward"]
M5["_micro_batch_forward(): 同左"]
end
subgraph ActorWorker["xtuner.v1.rl.trainer.actor_worker.ActorWorker"]
A1["mtp_config = model_cfg.mtp_config"]
A2["mtp_cfgs = [MTPLossConfig per depth]"]
A3["_mtp_trainable_parameters =<br/>trainable_parameters() 中 mtp_block.*"]
end
subgraph Engine["xtuner.v1.engine.train_engine.TrainEngine"]
E1["优化器 = trainable_parameters()<br/>(全部 requires_grad=True)"]
E2["DCP save: ignore_frozen_params=<br/>dcp_ignore_frozen_params(默认 True)"]
end
subgraph Loader["xtuner.v1.utils.loader.HFCheckpointLoader"]
L1["is_key_exist: key in weight_map"]
L2["load: 直接查 weight_map,<br/>缺失返回 None"]
end
subgraph WI["xtuner.v1.rl.weight_update.weight_iterator.WeightIterator"]
W1["iter_hf_batches: _load_spec_params()<br/>→ _get_hf_param → 发布全部参数"]
end
C1 --> M1 --> M2
C1 --> M3 --> M4
C1 --> M5
C1 --> A1 --> A2 --> A3
M1 --> E1
M2 --> E2
M1 -.发布.-> W1
L2 -.from_hf.-> M1
Main Flowchart after this PRgraph TD
subgraph MTPConfig["xtuner.v1.module.mtp.config.MTPConfig"]
C1[num_layers / share_weights /<br/>loss_type / detach_* ]
C2["frozen: bool = False ★新增字段"]
end
subgraph MoE["xtuner.v1.model.moe.moe.MoE"]
M1["__init__: build_mtp_block() 后<br/>frozen 则 mtp_block.requires_grad_(False) ★"]
MP["property _mtp_training_enabled<br/>= block 非空 且 config 非空 且 not frozen ★"]
M2["update_bias(): 仅 _mtp_training_enabled<br/>才消费 MTP 行 ★"]
M3["build_loss_ctx_batch(): 由<br/>mtp_config 非空改为 _mtp_training_enabled ★"]
M4["_forward(): 由 mtp_block 非空<br/>改为 _mtp_training_enabled ★"]
M5["_micro_batch_forward(): 同左 ★"]
end
subgraph ActorWorker["xtuner.v1.rl.trainer.actor_worker.ActorWorker"]
A1["mtp_config = model_cfg.mtp_config"]
A0["frozen 则 self.mtp_config = None ★<br/>(null object)"]
A2["mtp_cfgs = [](frozen 时为空)"]
A3["_mtp_trainable_parameters = []"]
end
subgraph Loader2["xtuner.v1.utils.loader.HFCheckpointLoader"]
L1["is_key_exist: 直接键 OR<br/>_resolve_fused_mtp_expert ★"]
L2["load: 直接键优先;缺失键走<br/>_load_fused_mtp_expert 切片 ★"]
L3["regex: mtp.layers.N.mlp.experts.M.<br/>(gate|up|down)_proj.weight"]
L4["gate = 前半, up = 后半;<br/>BF16/F16/F32 之外显式报错 ★"]
end
WI["WeightIterator: 不变(load spec 天然含 frozen 参数)"]
C2 --> M1
C2 --> MP
MP --> M2
MP --> M3
MP --> M4
MP --> M5
C2 --> A0 --> A2
M1 --> WI
L3 --> L1
L4 --> L2
classDef hl fill:#ffe08a,stroke:#b58900,stroke-width:2px;
class C2,M1,MP,M2,M3,M4,M5,A0,L1,L2,L4 hl;
修改与流程对应说明:
核心原理
核心实现与对应的单测
抽象与信息隐藏评估依据 improve-codebase-architecture / codebase-design 词汇(module/interface/depth/seam/locality/deletion test):
新增功能守则(R1–R3)
公开接口的线性业务流程评估
单测建议
其他 issuesX2292-F01 — 新文档未挂 toctree,成为 orphan 页面(轻微,已确认)
on_policy_distillation.md
native_frozen_mtp.mdX2292-F02 — 通用 loader 模块内联 MTP 键族知识(设计建议,已确认/推断)
class HFCheckpointLoader:
def __init__(...):
self._alias_resolvers: list[Callable[[str], AliasHit | None]] = [resolve_fused_mtp_expert]
def _resolve(self, key):
for r in self._alias_resolvers:
hit = r(key)
if hit: return hitX2292-F03 — 量化 fused MTP 源从"静默缺失"变为显式报错(行为变更,已确认,属预期)
X2292-F04 — frozen 时
|
| 检查项 | 结果 | 证据 |
|---|---|---|
| toctree 新增条目 | ✅ 已加入,位置合理(append 在 on_policy_distillation.md 之后) |
docs/en/rl/advanced_tutorial/index.rst:12 |
| 缩进一致性 | ✅ 3 空格缩进与既有条目一致 | index.rst:7-12 |
| 目标页存在 | ✅ docs/en/rl/advanced_tutorial/native_frozen_mtp.md 存在于 80a7270 树 | 工作树 ls 确认 |
| 生产代码零改动 | ✅ diff stat 仅 1 个 .rst 文件,无 .py/.yaml 变更 | git diff 10d89ad..80a7270 --stat |
| 文档自身内容 | 未变(本轮仅挂链) | diff 仅 +1 行 |
| zh_cn 翻译 | 未新增(用户明示不作门禁) | docs/zh_cn/rl/advanced_tutorial/ 无对应文件,记录备查 |
剩余观察(非阻塞)
- zh_cn 缺失(原 F01 第二部分):按用户指令本轮不作功能门禁;若后续要求双语一致,需补
docs/zh_cn/rl/advanced_tutorial/native_frozen_mtp.md并在 zh index.rst 挂链。状态:待办,非缺陷。 - 未执行 Sphinx 构建(只读约束);RST 语法为纯 toctree 条目追加,静态核对无 orphan 风险残留。
结论
- X2292-F01:已修复(已验证事实,index.rst:12 + 目标页存在)。
- 无新增问题。
- 原报告 X2292-F02~F08 维持原结论(设计建议/行为变更声明/契约提示),其中 F08 的 DCP 默认值 warning 仍为可选建议。
- 关系声明不变:本 PR 与 [Feature] Preserve external frozen MTP across RL updates and checkpoints #2289 互斥备选,不含 [Draft] Retain native Agent tasks across complete-turn weight publications #2257;GPU/集成证据边界同原报告(索引转述,非本树独立验收)。
报告生成:2026-11-11,审阅者:Kimi CLI 独立复审。
Keep native MTP weights in the actor while freezing their dedicated parameters, so existing load/save/weight-publication paths can continue supplying speculative decoding.
MTPConfig(num_layers=1, frozen=True)excludes dedicated MTP parameters from the optimizer and skips MTP training forward/loss; the main model and shared embedding/output head continue training. Defaultfrozen=Falsebehavior is preserved.Base: main
f1fddc506a49b7d0aa71554e26908885b37211cc. Unchanged code head:80a72703464e2c813def1a1b2b610df379b0cc05. This standalone PR does not include the Agent lifecycle stack (#2294–#2299), Lagent or recipe changes.Configuration and code entry points
MTPConfig(num_layers=1, frozen=True)in the native model.model_cfg.dcp_ignore_frozen_params=Falsefor a self-contained native DCP; the existing default can omit frozen parameters.lmdeploy_speculative_algorithm="qwen3_5_mtp"and a positive draft length in the Qwen setup. Freezing alone does not enable speculation or establish speedup.xtuner/v1/module/mtp/config.py,model/moe/moe.py,rl/trainer/actor_worker.py, andutils/loader.py. Feature documentation.Source and loading repair
The minimal native-freezing patch is extracted from xs1997zju/xtuner#3, fixed at
76f2dfdbecff27a10ca53e39e04c3079651282ea; source authorship is retained. Full-model validation also exposed Qwen3.5 HF checkpoints storing fused MTP expert tensors. A narrow safetensors alias/slice resolver maps those BF16/FP16/FP32 tensors to native per-expert load specs. Direct keys take precedence; malformed shape/index/dtype fails explicitly. Quantized fused-source aliases are unsupported; bin and non-MTP behavior is preserved.Validation and boundaries
GPU evidence was collected on explicitly recorded integrated candidates containing the corresponding native runtime patch, not by executing every exact standalone public branch tree. Cold restore evidence stops after restored state and a real response: no third optimizer update or independent elementwise comparison of all optimizer moments is claimed. No convergence, memory-superiority, speedup or general failure-recovery claim.
Delivery relationship
Current delivery selects native #2292 only. External-source #2289 is not included or selected; older historical comments/documents may describe it as an alternative. This update does not close or modify #2289.
The Agent lifecycle work is split from #2257 into #2294–#2299. Lagent #385 targets v1.0.0. CRG !78 provides the separate CyberGym recipe; its published recipe currently selects native wiring explicitly, while the separately tested local simplification constructs the native config directly. XTuner does not read that recipe's route-selection environment variable.
Developer guide with feature-to-PR mapping, diagrams, single-node scripts and quick-GPU method.
Keep Draft; no automatic merge. This publication update changes only title/description/comments, preserving the code head.