[Reproducibility] Training script / config for MiniMax-H3-TrainingAdapter (DeCFG adapter) itself

[kmatzen]

Hi, and thanks for open-sourcing the MiniMax-H3 work.

I'd like to reproduce the training adapter itself — https://modelscope.ai/models/DiffSynth-Studio/MiniMax-H3-TrainingAdapter — not the downstream LoRAs.

What I've already found in the repo

• examples/minimax_h3/model_training/train.py and the two-stage sft:data_process → sft:train workflow.
• The .sh scripts under examples/minimax_h3/model_training/lora/ (e.g. MiniMax-H3-Pruned-FL2VA.sh, the Ref2VA variants). These train downstream LoRAs and reference the adapter only as an optional --preset_lora_path / --preset_lora_model "dit" to fuse in during training.
• The base weights (Comfy-Org/MiniMax-H3, MiniMax/MiniMax-H3) and the MiniMax-H3-Self-Generated-Dataset.

What I can't find
A script or config that produces the adapter itself (the rank-64 DeCFG LoRA, FL2VA and Ref2VA variants) from the CFG-distilled base + the self-generated dataset. The example scripts consume the adapter rather than create it.

Could you share the following for the adapter's own training?

1. The exact training script / command (equivalent .sh) used to produce model_for_comfy_dit.safetensors and model_ref2va_for_comfy_dit.safetensors.
2. The DeCFG / "differential training" details — what the objective is and how it differs from standard SFT LoRA training (loss formulation, any CFG/DeCFG-specific handling, timestep sampling/shift).
3. Hyperparameters: LoRA rank (64?) and target modules, learning rate, batch size, number of steps/epochs, dataset_repeat, resolution / num_frames, audio_loss_weight, optimizer/schedule, and seed if fixed.
4. Which subset/split of MiniMax-H3-Self-Generated-Dataset was used, and whether the released dataset is the complete training set or a sample.
5. Base checkpoint used as the starting point (the pruned bf16 DiT, or another), plus any pre/post-processing of the adapter weights (e.g. the ComfyUI qkv layout conversion).

Happy to open a PR to add a reproduction script/README under examples/minimax_h3/ if that's easier on your side. Thanks!

[mi804]

Hi, thanks for the detailed questions!

1. Training scripts

The two-stage commands for training the FL2VA and Ref2VA adapters are available in the Training Adapter Details section of the [model card](https://www.modelscope.cn/models/DiffSynth-Studio/MiniMax-H3-TrainingAdapter). These commands train the adapters themselves. The *_for_comfy_dit.safetensors files are layout-converted versions of the resulting adapters, as explained below.

2. DeCFG / differential training

The adapter itself is trained using standard LoRA SFT on self-generated samples, with the usual flow-matching objective.

“Differential training” refers to the overall workflow: first train this adapter, then load it as a fixed preset while training a new downstream LoRA, and finally remove the adapter for inference while retaining the new LoRA.

3. Hyperparameters

Both adapters were trained on 8 NVIDIA A100 GPUs, using LoRA rank 64, targeting qkv_proj, out_proj, fc1, and fc2, with a configured learning rate of 1e-4, 124 frames, and dynamic resolution capped at 399,360 pixels (480P).

4. Training data

The training samples were randomly selected from the [released dataset](https://www.modelscope.cn/datasets/DiffSynth-Studio/MiniMax-H3-Self-Generated-Dataset): approximately 3,000 T2VA samples for the FL2VA adapter and 2,500 reference-conditioned samples for the Ref2VA adapter.

5. Base checkpoint and weight conversion

The published commands start from the official MiniMax/MiniMax-H3 checkpoints under FL2VA/transformer/model*.safetensors and Ref2VA/transformer/model*.safetensors.

For ComfyUI compatibility, the *_for_comfy_dit variants reorder the rows of the qkv_proj LoRA-B tensors from the official [head][role][dim] layout to [role][head][dim]. Every other tensor is byte-identical to its counterpart. Please use the adapter variant matching your DiT’s layout.