npu-adapter-reviewer

NPU Adapter Reviewer - GPU to Ascend NPU adaptation review expert

This is an Agent Skill specifically for adapting GPU code to Huawei Ascend NPU. This skill covers the complete adaptation workflow: code analysis, blocker identification, adaptation script writing, verification plan design, and final report generation.

Core Workflow

Phase 1: Code Repository Acquisition and Analysis

Task 1.1: Obtain Source Code

Based on the input provided by the user (local path or GitHub link), obtain the complete code repository:

# 如果是GitHub链接，先克隆
git clone <repo_url> /tmp/gpu_code_base
cd /tmp/gpu_code_base

If it is a local path, analyze it directly

ls -la <local_path>

**任务1.2：全面代码扫描**

使用并行探索的方式分析代码结构：

1. **探索Agent 1 - 代码结构分析**
   - 找出所有Python文件、CUDA文件、C++文件
   - 识别项目目录结构
   - 找出主要的入口文件和配置文件

2. **探索Agent 2 - GPU依赖识别**
   - 搜索CUDA API调用（`cudaMalloc`, `cudaMemcpy`, `kernel<<<...>>>`, `torch.cuda`等）
   - 搜索PyTorch GPU相关代码（`.cuda()`, `.to('cuda')`, `torch.device('cuda')`等）
   - 搜索TensorRT相关代码
   - 搜索深度学习框架特定API（Transformer引擎、Flash Attention等）

3. **探索Agent 3 - 外部库依赖**
   - 搜索`import`和`from ... import`语句
   - 识别所有第三方库依赖
   - 检查是否有NPU不支持的库

**任务1.3：生成代码结构报告**

输出以下信息：
- 项目总文件数、代码行数
- 文件类型分布（Python/CUDA/C++/其他）
- 主要依赖库列表
- 核心模块及其功能描述

### 阶段2：GPU到NPU迁移堵点识别

**任务2.1：算子兼容性分析**

逐类识别GPU专用算子在NPU上的兼容性：

| 堵点类别 | GPU典型实现 | NPU替代方案 | 迁移难度 |
|---------|------------|------------|---------|
| CUDA核心算子 | `__global__`, `__device__`函数 | Ascend C算子 / ATB | 高 |
| 内存操作 | `cudaMallocHost`, `cudaMallocManaged` | `aclrtMalloc`, `HI_MPI_MALLOC` | 中 |
| 流和事件 | `cudaStream_t`, `cudaEvent_t` | `aclrtStream`, `aclrtEvent` | 中 |
| cuBLAS/cuDNN | `cublasGemmEx`, `cudnnConvolutionForward` | `aclblasGemmEx`, 算子融合 | 高 |
| Flash Attention | `flash_attn_varlen_func` | 昇腾Flash Attention算子 | 中 |
| 自定义算子 | PyTorch CUDA扩展 | ATC/ACL算子 | 高 |
| AMP/混合精度 | `torch.cuda.amp` | `ascend_mixed_precision` | 低 |

**任务2.2：识别具体堵点**

对每个GPU API调用，生成以下分析：

Blocker ID: #001

• File Location: src/attention/cuda_impl.cu:142
• GPU API: cudaStreamCreate(&stream)
• NPU Replacement: aclrtCreateStream(&stream)
• Migration Plan:
  1. Replace header files aclrt.h
  2. Replace API calls
  3. Handle error code differences
• Estimated Effort: 0.5 person-days
• Impact Scope: Global stream management

**任务2.3：生成堵点清单**

输出完整的堵点列表，按影响范围和迁移难度排序。

### 阶段3：适配脚本编写

**任务3.1：创建NPU适配层**

根据识别的堵点，创建适配脚本：

1. **创建 `npu_compat.py` - Python层兼容适配**
   ```python
   # 自动检测运行设备
   def get_device():
       if is_npu_available():
           return "npu"
       elif is_cuda_available():
           return "cuda"
       else:
           return "cpu"
   
   # 替换torch.cuda调用
   def to_device(tensor):
       device = get_device()
       if device == "npu":
           return tensor.npu()
       elif device == "cuda":
           return tensor.cuda()
       return tensor

1. Create npu_ops.py - NPU operator wrapper

   • Wrap all CUDA core operators as NPU versions
   • Keep the original interface, implement NPU adaptation internally

2. Create build_npu.sh - Compilation script

   • ASCEND C operator compilation command
   • Dependency environment check
   • Error diagnosis

Task 3.2: Modify Existing Code

Generate the modified code files, keep the original files and create.npuVersion:

• Replace all GPU-specific calls
• Add device detection logic
• Add fallback mechanism

Phase 4: Verification Plan Design

Task 4.1: Create Verification Script

Based on the adaptation content, generate the verification script verify_npu.sh：

#!/bin/bash
# NPU适配验证脚本

echo "=== 1. 环境检查 ==="
check_npu_env() {
    # 检查NPU驱动
    ls -la /dev/*npu* 2>/dev/null || echo "Warning: NPU device not found"
    # 检查CANN
    echo $ASCEND_TOOLKIT_HOME
    # 检查Python包
    python3 -c "import torch; print('PyTorch version:', torch.__version__)"
    python3 -c "import torch_npu; print('torch_npu installed')"
}

echo "=== 2. 模块导入测试 ==="
test_imports() {
    cd <project_path>
    python3 -c "import npu_compat; print('npu_compat OK')"
    python3 -c "import npu_ops; print('npu_ops OK')"
}

echo "=== 3. 功能验证 ==="
test_functions() {
    # 运行基础测试
    python3 -m pytest tests/test_npu_*.py -v
    # 验证算子精度
    python3 scripts/verify_precision.py
}

echo "=== 4. 性能基准测试 ==="
benchmark() {
    python3 scripts/benchmark.py --device npu --compare cuda
}

Task 4.2: Precision Verification Script

Generate verify_precision.py：

import numpy as np

def verify_npu_precision(cuda_result, npu_result, rtol=1e-3, atol=1e-3):
    """验证NPU与GPU输出精度差异"""
    diff = np.abs(cuda_result - npu_result)
    max_diff = np.max(diff)
    mean_diff = np.mean(diff)
    
    passed = np.allclose(cuda_result, npu_result, rtol=rtol, atol=atol)
    return {
        "passed": passed,
        "max_diff": max_diff,
        "mean_diff": mean_diff,
        "rtol": rtol,
        "atol": atol
    }

Phase 5: Review Report Generation

Task 5.1: Generate Markdown Report

Based on the verification results, generate the complete review report:

# GPU到昇腾NPU适配审查报告
# CodeReview_Results_YYYY-MM-DD.md

## 1. 执行摘要

| 项目 | 内容 |
|-----|------|
| 原始代码仓库 | `<repo_url>` 或 `<local_path>` |
| 审查日期 | YYYY-MM-DD |
| 适配状态 | ✅ 完全适配 / ⚠️ 部分适配 / ❌ 适配失败 |
| 识别堵点总数 | XX个 |
| 已适配堵点 | XX个 |
| 剩余堵点 | XX个 |

## 2. 原始代码分析

### 2.1 代码结构概览
- 总文件数：XX
- Python代码行数：XX
- CUDA/C++代码行数：XX
- 核心模块：...

### 2.2 依赖分析
| 库名 | 版本 | NPU兼容性 | 替代方案 |
|-----|------|----------|---------|
| torch | 2.x | ✅ 兼容 | torch_npu |
| flash-attn | 2.x | ⚠️ 部分 | 昇顿Flash Attention |

## 3. 迁移堵点详细分析

### 3.1 算子兼容性问题

#### 问题 #001: CUDA Stream管理
- **文件**: `src/utils/stream_manager.py:45`
- **GPU API**: `cudaStreamCreate`
- **问题描述**: 使用CUDA流管理异步执行
- **NPU替代**: `aclrtCreateStream`
- **影响范围**: 全局，影响所有异步操作
- **迁移建议**: 
  ```python
  # 修改前
  import torch.cuda
  stream = torch.cuda.Stream()
  
  # 修改后
  import torch_npu
  stream = torch.npu.Stream()

• Status: ✅ Adapted / ⚠️ Pending

Issue #002: Flash Attention operator

• File: src/attention/flash_attn_impl.py:78
• GPU API: flash_attn_varlen_func
• Issue Description: Use Flash Attention to accelerate attention computation
• NPU Replacement: Ascend flash_attn operator or MindSpore flash_attention
• Impact Scope: High, core inference performance
• Migration Recommendation:

  # 修改前
  from flash_attn import flash_attn_func
  output = flash_attn_func(q, k, v)
  
  # 修改后
  # 方案1: 使用torch_npu的算子
  import torch_npu
  output = torch_npu.npu_flash_attention(q, k, v)
  
  # 方案2: 使用ATB库
  from ascend_toolkit import flash_attention
  output = flash_attention(q, k, v)
  

• Status: ✅ Adapted / ⚠️ Pending

3.2 Model Loading and Weight Management Issues

Issue #003: GPU weight format

• File: src/model/loader.py:112
• Issue Description: Weights are stored in CUDA format, direct loading will fail
• Migration Recommendation:

  # 修改前
  state_dict = torch.load(weights_path)
  model.load_state_dict(state_dict)
  
  # 修改后
  state_dict = torch.load(weights_path, map_location='cpu')
  # 转换权重
  for k, v in state_dict.items():
      if isinstance(v, torch.Tensor):
          state_dict[k] = v.npu()
  model.load_state_dict(state_dict)
  

• Status: ✅ Adapted

3.3 Computational Performance Bottlenecks

Issue #004: Missing operator fusion

• File: src/model/inference.py:89
• Problem Description: Multiple independent operators cause performance degradation
• Migration Recommendation: Use ATC for operator fusion optimization
• Estimated Performance Improvement: 20-30%
• Status: ⚠️ Pending

3.4 NPU Memory and KV Cache Management

Issue #005: Dynamic Memory Allocation

• File: src/cache/kv_cache.py:56
• Problem Description: Using CUDA dynamic memory allocation
• Migration Recommendation: Use a fixed memory pool
• Status: ⚠️ Pending

3.5 Python-C++ Boundary Issues

Issue #006: C++ Extension Compilation

• File: src/utils/gpu_ext.cpp:145
• Problem Description: CUDA C++ extensions need to be recompiled
• Migration Recommendation: Rewrite using Ascend C or use ATB
• Status: ⚠️ Pending

3.6 Concurrency and Asynchrony Issues

Issue #007: Multi-stream Concurrency

• File: src/server/request_handler.py:78
• Problem Description: Using CUDA streams to achieve concurrency
• Migration Recommendation: Refactor to process-level concurrency
• Status: ⚠️ Pending

3.7 Configuration and Maintainability Issues

Issue #008: Hardcoded Device

• File: src/config.py:23
• Problem Description: Hardcoded in configurationcuda:0
• Migration Recommendation: Change to device detection
• Status: ✅ Adapted

4. Adapted Code List

4.1 New Files

File Name	Function	Status
npu_compat.py	Device detection and compatibility layer	✅
npu_ops.py	NPU operator wrapper	✅
build_npu.sh	Build script	✅
verify_npu.sh	Verification script	✅

4.2 Modified Files

File Name	Modification Details	Status
src/attention/flash_attn.py	Replace with NPU operators	✅
src/model/loader.py	Add weight conversion	✅
src/utils/stream_manager.py	Stream adaptation	✅

5. Verification Results

5.1 Environment Verification

• NPU driver installed
• CANN Toolkit configured
• torch_npu installed
• Python module importable

5.2 Functional Verification

• Basic module import test passed
• Device detection function works properly
• Forward inference executed successfully
• Weight loading and conversion works properly

5.3 Accuracy Verification

• Inference result difference from GPU < 0.1%
• Performance test pending (requires NPU hardware)

5.4 Issue Summary

Issue Type	Count	Severity
Resolved	XX	-
Unresolved	XX	High/Medium/Low

6. Adaptation Guide

6.1 Prerequisites

# 1. 安装CANN Toolkit
# 下载地址: https://www.hiascend.com/software/aiengine

# 2. 安装torch_npu
pip install torch torch_npu

# 3. 验证安装
python3 -c "import torch; import torch_npu; print('NPU available:', torch_npu.is_npu_available())"

6.2 Quick Adaptation Steps

Step 1: Clone and enter the project

git clone <repo_url>
cd <project_name>

Step 2: Install dependencies

pip install -r requirements-npu.txt

Step 3: Run verification

bash verify_npu.sh

Step 4: Execute inference

python3 run_npu.py --model <model_path> --input <input_data>

6.3 Common Troubleshooting

Issue	Cause	Solution
Import failed	CANN is not installed correctly	Reconfigure the environment variables
Operator not supported	The NPU does not support this operator	Use ATB as a replacement or develop a custom operator
Out of memory	Batch size too large	Reduce batch_size
Precision does not meet the standard	Mixed precision configuration issue	Check the AMP configuration

7. Recommendations for Follow-up Work

7.1 Short term (within 1 week)

• Complete the adaptation of the remaining blockers
• Conduct performance testing on real NPU hardware
• Optimize operator fusion

7.2 Medium term (within 1 month)

• Improve the error handling mechanism
• Add logging and monitoring
• Performance tuning

7.3 Long term

• Continuously follow CANN updates
• Automated testing process
• Improve documentation

---

Report generation time: YYYY-MM-DD HH:mm:ss Adaptation Engineer: AI Agent (NPU Adapter Reviewer) Report version: v1.0

**任务5.2：输出报告**

将报告保存到当前目录：

CodeReview_Results_YYYY-MM-DD.md

## 输出要求

1. **报告格式**: 必须是Markdown格式
2. **文件命名**: `CodeReview_Results_运行当天的日期.md`（格式：YYYY-MM-DD）
3. **保存位置**: 当前工作目录
4. **内容完整性**: 必须包含上述所有章节

## 特殊处理规则

### 如果验证完全通过
- 输出"适配成功"的状态
- 提供完整的适配指南
- 包含端到端运行说明

### 如果验证未完全通过
- 详细说明每个失败项
- 提供具体的修复建议
- 给出修改后的代码
- 标注需要人工介入的部分

## 知识参考

在执行过程中，可参考以下资料（按需加载）：
- `references/ascend_npu_best_practices.md` - 昇腾NPU最佳实践
- `references/cann_migration_guide.md` - CANN迁移指南
- `references/npu_python_api.md` - NPU Python API参考

请使用此skill完成GPU到昇腾NPU的完整适配审查工作。