mindspeed-llm-weight-prep

MindSpeed-LLM Weight Conversion

This Skill guides users in converting model weights between HuggingFace and Megatron formats, including LoRA weight merging and parallel sharding configuration.

Quick Start

HuggingFace → Megatron

python convert_ckpt.py \
    --use-mcore-models \
    --model-type GPT \
    --load-model-type hf \
    --save-model-type mg \
    --target-tensor-parallel-size 1 \
    --target-pipeline-parallel-size 4 \
    --add-qkv-bias \
    --load-dir ./model_from_hf/Qwen2.5-7B-Instruct/ \
    --save-dir ./model_weights/qwen25_7b_mcore/ \
    --tokenizer-model ./model_from_hf/Qwen2.5-7B-Instruct/tokenizer.json \
    --model-type-hf llama2 \
    --params-dtype bf16

Megatron → HuggingFace

python convert_ckpt.py \
    --use-mcore-models \
    --model-type GPT \
    --load-model-type mg \
    --save-model-type hf \
    --target-tensor-parallel-size 1 \
    --target-pipeline-parallel-size 1 \
    --load-dir ./model_weights/qwen25_7b_mcore/ \
    --save-dir ./model_from_hf/Qwen2.5-7B-Instruct/ \
    --tokenizer-model ./model_from_hf/Qwen2.5-7B-Instruct/tokenizer.json \
    --model-type-hf llama2 \
    --params-dtype bf16

When using v1 for MG→HF, TP=PP=1 must be set; v2 requires no parallel parameter settings (handled automatically). v1's --save-dir must point to the original HF model path (containing config.json and tokenizer). Output is saved in --save-dir under the mg2hf/ subdirectory.

Conversion Direction

Direction	Script	Purpose
HF → Megatron	convert_ckpt.py / convert_ckpt_v2.py	Before training: convert HF weights to Megatron format
Megatron → HF	convert_ckpt.py / convert_ckpt_v2.py	After training: convert Megatron weights back to HF format
LoRA merge and export	convert_ckpt.py(v1 only)	Convert after merging LoRA + base weights

Key Parameters

Parameter	Description	Required
--load-model-type	Source format:hf or mg	Yes
--save-model-type	Target format:hf or mg	Yes
--target-tensor-parallel-size	Tensor parallelism (TP)	Yes
--target-pipeline-parallel-size	Pipeline parallelism (PP)	Yes
--target-expert-parallel-size	Expert parallelism (EP, MoE models)	No (default 1)
--use-mcore-models	Enable Megatron-Mcore format	Yes (v1 only)
--load-dir	Source model path	Yes
--save-dir	Output path	Yes
--model-type-hf	HF model type	No (v1 default llama2, v2 default qwen3）
--tokenizer-model	Tokenizer file path	Yes
--params-dtype	Weight precision:fp16、bf16	Yes (v1 only)
--add-qkv-bias	Add QKV bias (required by some models)	Depends on model

TP/PP Configuration Rules

Rule	Description
TP × PP ≤ number of NPUs	Parallelism cannot exceed available NPUs
Number of layers divisible by PP	Otherwise, --num-layer-list dynamic allocation must be used
For MG→HF, v1 requires TP=PP=1	v2 requires no settings (handled automatically)
TP/PP must match during training	TP/PP during conversion must be consistent with the training script

Dynamic Pipeline Parallelism

When the number of layers is not divisible by PP:

# 14 层模型，PP=4：按 3,4,4,3 分配
--target-pipeline-parallel-size 4 \
--num-layer-list 3,4,4,3

LoRA Weight Handling

Convert after merging LoRA (v1 only)

python convert_ckpt.py \
    --use-mcore-models \
    --model-type GPT \
    --load-model-type mg \
    --save-model-type hf \
    --load-dir ./model_weights/base_mcore/ \
    --save-dir ./model_from_hf/merged_output/ \
    --lora-load ./lora_output/ \
    --lora-r 8 \
    --lora-alpha 16 \
    --lora-target-modules linear_qkv linear_proj linear_fc1 linear_fc2 \
    --target-tensor-parallel-size 1 \
    --target-pipeline-parallel-size 1 \
    --tokenizer-model ./model_from_hf/tokenizer.json

LoRA Parameter Matching

During LoRA conversion, --lora-r、--lora-alpha、--lora-target-modules must be consistent with training。

Parameter	Description
--lora-r	LoRA rank (e.g., 8, 16)
--lora-alpha	LoRA scaling factor (e.g., 16, 32)
--lora-target-modules	Target modules:linear_qkv linear_proj linear_fc1 linear_fc2

v1 vs v2 Comparison

Feature	v1 (convert_ckpt.py)	v2 (convert_ckpt_v2.py)
LoRA conversion	Supported	Not supported
Large model memory optimization	General	Streaming load, TB-level parameters do not crash
MoE advanced features	Basic	--moe-grouped-gemm、--moe-tp-extend-ep
Default HF type	llama2	qwen3
MTP layer	Not supported	--mtp-num-layers
Reduced layer debugging	Not supported	--num-layers + --first-k-dense-replace

Note: v1 and v2 are independent systems; do not mix them (e.g., use v2 for HF→MG, then use v1 for MG→HF).

Example scripts

The scripts are located in examples/mcore/<model_name>/：

Script mode	Purpose
ckpt_convert_<model>_hf2mcore.sh	HF → Megatron
ckpt_convert_<model>_mcore2hf.sh	Megatron → HF

# 示例：Qwen2.5-7B HF→Megatron
bash examples/mcore/qwen25/ckpt_convert_qwen25_7b_hf2mcore.sh

# 示例：LLaMA-3-8B HF→Megatron
bash examples/mcore/llama3/ckpt_convert_llama3_8b_hf2mcore.sh

Common issues

Q: After conversion, training reports weight loading failure

Confirm that the TP/PP during conversion is completely consistent with the settings in the training script.

Q: OOM when converting large models (100B+)

Use the v2 converter:python convert_ckpt_v2.py, which supports streaming load.

Q: MG→HF output is missing config.json and tokenizer

MG→HF only outputs weight files (saved in the mg2hf/ subdirectory). config.json, tokenizer, etc. need to be copied from the original HF model directory.

Q: LoRA precision is abnormal after conversion

Qwen models may have a dtype mismatch (a bf16 model produces fp16 weights after merging). Check the torch_dtype field in config.json.

Usage order

After weight conversion is complete:

1. Training startup → Use mindspeed-llm-training

Reference resources

• Detailed conversion guide - complete v1/v2 parameters, MoE conversion, advanced usage
• MindSpeed-LLM repository