SuperAPI AI Directory
Partner with us中文AI newsFavorites
Skillsmindspeed-llm-env-setup

mindspeed-llm-env-setup

Agents & models

View repository

Updated 10-15 17:19

About this skill

MindSpeed-LLM Ascend NPU Environment Setup

This Skill guides users through setting up a MindSpeed-LLM distributed training environment on Huawei Ascend NPU.

Component Relationships

Megatron-LM (NVIDIA)     ← 分布式训练核心 (TP/PP)，使用 core_v0.12.1 分支
    ↑
MindSpeed (Huawei)       ← 昇腾适配层，猴子补丁优化 Megatron 内核
    ↑
MindSpeed-LLM (Huawei)   ← 应用层：训练脚本、数据预处理、权重转换

Quick Start

Complete the environment setup in 6 steps:

# 1. 激活 CANN 环境（CANN 8.5.0+ 路径；旧版用 /usr/local/Ascend/ascend-toolkit/set_env.sh）
source /usr/local/Ascend/cann/set_env.sh
source /usr/local/Ascend/nnal/atb/set_env.sh

# 2. 安装 PyTorch + torch_npu
pip install torch==2.7.1 --index-url https://download.pytorch.org/whl/cpu
pip install torch_npu==2.7.1rc1
pip install numpy pyyaml scipy attrs decorator psutil

# 3. 克隆并安装 MindSpeed
git clone https://gitcode.com/ascend/MindSpeed.git
cd MindSpeed && pip install -r requirements.txt && pip install -e . && cd ..

# 4. 克隆 Megatron-LM 并复制核心模块
git clone https://github.com/NVIDIA/Megatron-LM.git
cd Megatron-LM && git checkout core_v0.12.1 && cd ..
cp -r Megatron-LM/megatron MindSpeed-LLM/

# 5. 安装 MindSpeed-LLM
cd MindSpeed-LLM && pip install -r requirements.txt

# 6. 验证环境
python -c "
import torch
import torch_npu
print(f'NPUs available: {torch_npu.npu.device_count()}')
print(f'NPU ready: {torch.npu.is_available()}')
"

Version Compatibility Matrix

CANN PyTorch torch_npu Python Megatron-LM
8.5.0 2.7.1 2.7.1rc1 3.10 core_v0.12.1

The above are the current master branch versions. For historical versions, please refer to Official Installation Documentation. Check CANN version:cat /usr/local/Ascend/cann/latest/aarch64-linux/ascend_toolkit_install.info(CANN 8.5.0+) or cat /usr/local/Ascend/ascend-toolkit/latest/aarch64-linux/ascend_toolkit_install.info(older version).

Docker Container Creation

docker run -it --name mindspeed-llm \
    --device /dev/davinci0 \
    --device /dev/davinci1 \
    --device /dev/davinci2 \
    --device /dev/davinci3 \
    --device /dev/davinci4 \
    --device /dev/davinci5 \
    --device /dev/davinci6 \
    --device /dev/davinci7 \
    --device /dev/davinci_manager \
    --device /dev/devmm_svm \
    --device /dev/hisi_hdc \
    -v /usr/local/dcmi:/usr/local/dcmi \
    -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \
    -v /usr/local/Ascend/driver/lib64:/usr/local/Ascend/driver/lib64 \
    -v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info \
    -v /usr/local/sbin/npu-smi:/usr/local/sbin/npu-smi \
    -v /home/workspace:/home/workspace \
    swr.cn-south-1.myhuaweicloud.com/ascendhub/cann:8.5.0-910b-ubuntu22.04-py3.10 \
    bash

Note: The CANN container image does not include ML dependencies. After entering the container, you need to manually install PyTorch, torch_npu, etc.

Directory Structure

After installation is complete, the workspace structure should be as follows:

workspace/
├── MindSpeed/                    # 加速库
├── MindSpeed-LLM/                # 主项目
│   ├── megatron/                 # 从 Megatron-LM 复制的核心模块
│   ├── examples/mcore/           # 各模型的示例脚本
│   ├── preprocess_data.py        # 数据预处理入口
│   ├── convert_ckpt.py           # 权重转换 v1
│   ├── convert_ckpt_v2.py        # 权重转换 v2
│   ├── pretrain_gpt.py           # 预训练入口
│   └── posttrain_gpt.py          # 微调/后训练入口
├── Megatron-LM/                  # NVIDIA 原始仓库（仅用于复制 megatron/）
├── model_from_hf/                # HuggingFace 模型权重
└── dataset/                      # 训练数据集

Environment Verification Checklist

Check Item Command Expected Result
CANN Environment npu-smi info Display NPU device information
PyTorch python -c "import torch; print(torch.__version__)" 2.7.1
torch_npu python -c "import torch_npu; print(torch_npu.__version__)" 2.7.1rc1
NPU Count python -c "import torch_npu; print(torch_npu.npu.device_count())" ≥1 (consistent with hardware)
MindSpeed pip show mindspeed Display package information
megatron module ls MindSpeed-LLM/megatron/ Directory exists and is non-empty

Common Issues

Q: ModuleNotFoundError: No module named 'yaml' When importing torch_npu

CANN Docker image is missing basic Python packages:

pip install numpy pyyaml scipy attrs decorator psutil

Q: Network issues cause pip install or git clone to fail

Use a proxy:

export http_proxy=http://127.0.0.1:7890
export https_proxy=http://127.0.0.1:7890
pip install torch==2.7.1 --index-url https://download.pytorch.org/whl/cpu

Q: Megatron-LM checkout branch not found

Confirm you are using the correct branch name (note the core_v prefix):

cd Megatron-LM
git branch -a | grep core
git checkout core_v0.12.1

Usage Order

After the environment setup is complete, proceed with training deployment in the following order:

  1. Data Preprocessing → Use mindspeed-llm-data-prep
  2. Weight Conversion → Use mindspeed-llm-weight-prep
  3. Training Launch → Use mindspeed-llm-training

Reference Resources