SuperAPI AI Directory
Partner with us中文AI newsFavorites
Skillsexternal-cannbot-ops-ops-profiling

external-cannbot-ops-ops-profiling

Deployment & operations

View repository

Updated 10-15 17:19

About this skill

On-board performance collection and tuning

Collect operator performance data on a real NPU, systematically interpret the 8 CSV metric files, determine whether performance meets standards, locate the bottleneck type, and provide actionable optimization suggestions.

Applicable scenarios

Scenario Description
Performance acceptance after operator development is completed Confirm that the operator reaches the expected performance level
Performance issue localization Accurately locate bottlenecks through CSV metrics
Optimization effect verification Compare CSV data before and after optimization
Agent team testing phase Called by tester/developer for automated performance analysis

Workflow

Step 1: 构建算子
    ↓
Step 2: msprof op 采集数据
    ↓
Step 3: 归档数据 + 生成统计摘要
    ↓
Step 4: 读摘要 → 对照性能标准判定（需要时读原始 CSV）
    ↓
Step 5: 如不达标，查阅瓶颈优化速查表
    ↓
Step 6: 修改代码 → 回到 Step 2 重新采集（数据自动归档为新一轮）

Step 1: Build the operator

Direct-call operator:

cd operators/{operator_name} && mkdir -p build && cd build && cmake .. && make -j

aclnn operator:

bash build.sh --pkg --soc=ascend910b --ops={operator_name} --vendor_name=custom -j16
./build_out/*.run --install-path=$CANN
bash build.sh --run_example {operator_name} eager cust --vendor_name=custom

Step 2: msprof op collection

# 基本用法
msprof op ./demo

# 推荐用法（含预热 + 指定输出）
msprof op --warm-up=10 --output=./msprof_output ./demo

# 多次运行取均值
msprof op --warm-up=10 --launch-count=5 --output=./msprof_output ./demo

Key parameters:

Parameter Description When to use
--warm-up=N Warm up N times before collection Always recommended, to avoid DVFS (dynamic frequency scaling) affecting the first run
--launch-count=N Run N times and take the average When statistical stability is needed
--output=<dir> Specify the output directory Avoid scattered results
No need for --soc-version On-board automatic hardware detection —

Output: Generate in the specified directory or the current directory OPPROF_{timestamp}_XXX/ folder.

Step 3: Archive data + generate statistical summary

# 找到最新 OPPROF 目录
OPPROF_DIR=$(ls -td <output_dir>/OPPROF_* | head -1)

# 归档 CSV + 生成摘要（自动创建 docs/perf/round_NNN/）
python3 {skill_path}/scripts/perf_summary.py $OPPROF_DIR operators/{operator_name}

The script will automatically:

  1. In operators/{operator_name}/docs/perf/round_NNN/ create an archive directory (rounds automatically increment)
  2. Copy all 8 original CSV files to the archive directory
  3. Generate summary.txt statistical summary (min/avg/max of all metrics,no judgment performed）

Agent analysis flow：

  1. First read summary.txt — obtain a global overview (about 30 lines of compact text)
  2. Combine with references/csv_fields_reference.md — understand the meaning and thresholds of each metric
  3. Read the original CSV when anomalies are found — e.g., inter-core imbalance, Read PipeUtilization.csv to view per-core data
  4. Combine with references/optimization_quickref.md — locate the bottleneck type and optimization methods

Important：summary.txt It is only statistical aggregation, and does not include analytical judgment. All bottleneck determinations and optimization suggestions are completed autonomously by the Agent based on the criteria in Step 4 and Step 5. The original CSV files are fully retained in the same directory and can be Read at any time.

Step 4: Performance standard determination

Determine whether the operator performance meets the standard according to the table below.Performance meets standard = the actual elapsed time of the longest pipeline is close to the theoretical elapsed time。

4.1 Overall determination flow

读取 OpBasicInfo.csv → 获取 Task Duration 和 Block Dim
    ↓
读取 PipeUtilization.csv → 找到各流水占比最高的单元
    ↓
计算理论耗时（搬运量/带宽 或 计算量/算力）
    ↓
比较实际耗时 vs 理论耗时
    ├── 差距 <20% → 性能达标（已接近硬件极限）
    ├── 差距 20-50% → 有优化空间，查阅瓶颈优化表
    └── 差距 >50% → 严重瓶颈，必须优化

4.2 Standard for each metric

Metric Standard condition Warning condition Severe issue
Inter-core load balancing Each core ai*_time(us) difference <10% difference 10-30% Difference >30%
Block Dim Equal to available cores (910B: 20–40 cores) Much smaller than available cores Block Dim = 1
VEC ratio Matches the operator type (see 4.3) VEC ratio >80% VEC ratio >90% with no room for optimization
MTE2 ratio <30% (compute-bound operator) 30-50% >50% (transfer becomes the bottleneck)
fixpipe_ratio <5% 5-15% >15% (address not aligned)
icache_miss_rate <5% 5-15% >15% (code size too large)
Total bank conflict share aiv_vec_total_cflt_ratio <5% 5-15% >15%
Total L2 Cache hit rate >80% 50-80% <50%
Head overhead <10% of total elapsed time 10-30% >30%
DoubleBuffer effect MTE2/VEC overlap >30% Overlap 10–30% Overlap <5%
Bandwidth utilization bw_usage_rate >60% 30-60% <30%

4.3 Expected ratio distribution for different operator types

Operator type Dominant pipeline Expected ratio Anomaly signal
Elementwise（Add/Mul/Relu） VEC vec_ratio 50-80% MTE2 ratio > VEC ratio
Reduction（ReduceSum/Max） VEC vec_ratio 40-70% scalar_ratio >20%
Activation（Softmax/Gelu） VEC vec_ratio 60-85% A large number of cast instructions
MatMul CUBE cube_ratio 40-70% vec_ratio > cube_ratio
Pure transfer（Transpose/Concat） MTE2/MTE3 mte2+mte3 combined >50% VEC ratio >30%

4.4 Theoretical elapsed time calculation

Theoretical transfer elapsed time:

理论耗时(us) = 搬运数据量(Byte) / GM 峰值带宽

Atlas A2: GM peak bandwidth approximately 1.8 TB/s = 1.8e12 Byte/s

Example: theoretical elapsed time for transferring a float-type 4096x4096 matrix = 4 * 4096 * 4096 / 1.8e12 = 37.28 us

Theoretical computation elapsed time:

理论耗时(us) = 计算元素数 / 理论算力

Atlas A2: float32 Vector peak compute power approximately 11.06 TOPS

Example: theoretical elapsed time for a single 32K float instruction computation = 32000 / 11.06e12 = 0.003 us

Decision logic:

  • Actual MTE2 elapsed time ≈ theoretical transfer elapsed time → MTE2 has reached its upper limit; the optimization direction is pipeline orchestration
  • Actual MTE2 elapsed time >> theoretical transfer elapsed time → MTE2 has not reached its upper limit; check alignment and transfer granularity
  • Actual VEC elapsed time ≈ theoretical computation elapsed time → VEC has reached its upper limit; room for optimization is limited
  • Task Duration ≈ longest pipeline elapsed time → pipeline orchestration is good; the other pipelines have been masked

Step 5: Bottleneck localization and optimization

After confirming the bottleneck type, consult references/optimization_quickref.md for specific optimization methods.

Quick lookup:

Bottleneck type Decision condition Preferred optimization
VEC Bound aiv_vec_ratio highest UB fusion, reduce Cast, fuse instructions
MTE2 Bound ai*_mte2_ratio highest Increase transfer granularity to ≥16KB, 512B alignment, L2 CacheMode
CUBE Bound aic_cube_ratio highest L0C accumulation, L1 data reuse
SCALAR Bound ai*_scalar_ratio >30% Reduce TilingData, reduce the number of cores
Inter-core imbalance Elapsed time difference between cores >10% Adjust the Tiling splitting strategy
Bank Conflict vec_bank_cflt_ratio >5% Adjust UB address, add padding
Large head overhead Head overhead share >30% Reduce the number of cores, reduce TilingData, externalize TPipe
DoubleBuffer not in effect No MTE2/VEC overlap Check whether InitBuffer sets bufNum=2
Pipeline bubble Multiple units all 30–50%, none dominant Increase the number of workspace copies, asynchronous iteration

Step 6: Verify the optimization effect

After each optimization, rerun Step 2 + Step 3. The data is automatically archived as round_NNN+1。

Comparison Method：

# 对比两轮摘要
diff operators/{operator_name}/docs/perf/round_001/summary.txt operators/{operator_name}/docs/perf/round_002/summary.txt

# 或直接读两个 summary.txt 进行对比分析

Comparison Points:

  1. Whether Task Duration decreases
  2. Whether the ratio of the bottleneck unit improves
  3. Whether inter-core balance improves (aiv_time min/max gap)
  4. Whether a new bottleneck is introduced

Data Directory Structure

msprof Output (temporary)

OPPROF_{timestamp}_XXX/         # msprof op 直接输出（临时目录）
├── dump/                       # 原始性能数据（无需关注）
├── OpBasicInfo.csv             # 算子基本信息（名称、核数、总耗时、频率）
├── PipeUtilization.csv         # 各流水线单元耗时和占比（最重要）
├── ArithmeticUtilization.csv   # Cube/Vector 指令 cycle 占比和计算量
├── Memory.csv                  # 内存读写带宽和数据搬运量
├── MemoryL0.csv                # L0A/L0B/L0C 读写带宽
├── MemoryUB.csv                # UB 读写带宽（Vector/Scalar）
├── L2Cache.csv                 # L2 Cache 命中率
├── ResourceConflictRatio.csv   # Bank conflict 和资源冲突占比
└── visualize_data.bin          # MindStudio Insight 可视化文件

Archive Directory (persistent)

operators/{算子名}/docs/perf/
├── round_001/                  # 第一轮采集（基线）
│   ├── OpBasicInfo.csv         # 完整原始 CSV（从 OPPROF 复制）
│   ├── PipeUtilization.csv
│   ├── Memory.csv
│   ├── ResourceConflictRatio.csv
│   ├── L2Cache.csv
│   ├── ArithmeticUtilization.csv
│   ├── MemoryUB.csv
│   ├── MemoryL0.csv
│   └── summary.txt            # 统计摘要（min/avg/max，不含判定）
├── round_002/                  # 第二轮（优化后对比）
│   ├── *.csv
│   └── summary.txt
└── ...                         # 自动递增

Complete field descriptions for each CSV file → references/csv_fields_reference.md

On-Device vs Simulation Selection

Dimension On-Device (msprof op) Simulation (msprof op simulator)
Requires NPU Yes No
Timing Precision Real hardware timing Cycle-level model estimation
Output 8 CSV files CSV + trace.json
Instruction-level pipeline diagram Requires additional parameters or separate simulation Default output
Resource conflict data Available（ResourceConflictRatio.csv） None
L2 Cache Real hit rate Estimated
DVFS impact Yes (requires warm-up) No
Suitable stage Performance acceptance, production tuning Early development, instruction-level debugging

Recommendation: Use simulation for rapid iteration during development, and use on-device to confirm real performance during acceptance.

Notes

  1. Must warm up: The first run is affected by DVFS and takes longer. Always use --warm-up=10
  2. Frequency check: Read OpBasicInfo.csv of Current Freq and Rated Freq. If Current < Rated, it indicates that the chip is not running at full frequency
  3. MTE2/MTE3 bandwidth sharing: When reading and writing GM at the same time, the total bandwidth is shared, and the theoretical time consumption should be calculated as (MTE2搬运量 + MTE3搬运量) / GM带宽 calculated
  4. Small data volume scenario: When the data volume is very small, the proportion of overhead will be very high. This is not necessarily an operator problem, but insufficient data volume
  5. Multi-core access to the same address: Multiple cores reading the same 512B address range at the same time will be serialized, causing abnormal MTE2 time consumption

Reference Resources

File Content When to consult
references/csv_fields_reference.md Complete field definitions and thresholds for the 8 CSV files During Step 4 analysis, when specific field meanings need to be understood
references/optimization_quickref.md Specific optimization methods and cases for each bottleneck type After locating the bottleneck in Step 5, look up optimization methods
scripts/perf_summary.py Statistical summary generation + CSV archiving Automatic archiving and summary generation in Step 3