external-gitcode-ascend-ascendc-operator-performance-optim

Ascend C operator performance optimization (troubleshoot → modify → verify closed loop)

This skill not only troubleshoots performance issues, but is also responsible for modifying code and verifying the optimization effect. The complete process is:

Phase 1: 排查 — 审查代码 + 学习设计文档，发现优化点
Phase 2: 基线 — 保存当前性能测试结果（自定义算子 vs 标杆）
Phase 3: 优化 — 学习 code-gen 知识后修改算子代码
Phase 4: 精度 — 精度验证（确保优化后功能正确）
Phase 5: 性能 — 同 case 性能对比（优化后 vs 标杆）
Phase 6: 迭代 — 未提升则继续优化，最多 3 轮

Phase 1: Troubleshoot — discover optimization points

1.1 Study the operator design document

MANDATORY — you must first understand the operator design before troubleshooting：

Read ascend-kernel/csrc/ops/<op_name>/design.md(if it exists), and extract:
- Operator type (elementwise / row processing / Cube)
- Tiling strategy (inter-core splitting / intra-core splitting)
- UB space allocation scheme
- Computation logic and data flow
Read op_host/<op_name>.cpp and op_kernel/<op_name>.cpp all source code

1.2 Troubleshoot stage by stage

Review the operator code stage by stage in the following order. For each stage, load the corresponding reference file and check the code item by item against it.

- [ ] 1. Tiling    — 数据在多核与 L2Cache 间的切分策略
- [ ] 2. 搬运      — DataCopy 的带宽利用率
- [ ] 3. API 使用  — Ascend C API 的高效用法
- [ ] 4. 内存      — 数据在存储层级中的放置策略
- [ ] 5. 流水      — CopyIn / Compute / CopyOut 的重叠执行

Each stage has an independent reference file; during troubleshooting,load only the current stage'sfile:

1. Tiling

Detailed example:references/tiling-prof.md

Check items:

1.1 Multi-core splitting：blockDim Is it set to the number of hardware cores?
- Coupled architecture:GetCoreNumAiv() or GetCoreNumAic()
- Separated architecture Vector operator: number of AIV cores (e.g., 40)
- Separated architecture Cube operator: number of AIC cores (e.g., 20)
- Separated architecture MIX operator: number of physical core groups (e.g., 20 = 40 AIV / 2), must not exceed the number of physical cores
1.2 L2Cache splitting: when 输入 + 输出 > L2Cache 容量 , is the data split into blocks by L2Cache size, with all cores cooperatively processing the same block before switching to the next block?
1.3 Inter-core load balancing: after L2Cache splitting, is the tail block assigned alternately among passes, to avoid certain cores always being left with the tail?

2. Data movement

Detailed example:references/data-copy-prof.md

Check items:

2.1 Single transfer size >= 16 KB: does each DataCopy transfer at least 16 KB? Below this value, bandwidth utilization drops significantly.
2.2 GM address 512B alignment: is the GM start address 512-byte aligned? (On the Atlas A2 series, 32B alignment has up to 30% lower bandwidth than 512B alignment.)
2.3 stride parameters instead of for loops: does strided data movement use DataCopyParams (blockCount/blockLen/srcStride/dstStride) issued in one call, rather than moving row by row using a for loop?

3. API usage

Detailed example:references/api-usage-prof.md

Check items:

3.1 TPipe created outside the kernel class：TPipe Is it created in the kernel entry function and passed into the class as a pointer? (A TPipe inside the class prevents Scalar constant folding, increasing scalar_time by about 17%.)
3.2 Pure data movement operators use TQueBind: for operators without Vector computation, is TQueBind<VECIN, VECOUT> used to replace the separate TQue<VECIN> + TQue<VECOUT>? (eliminate redundant DataCopy between LocalTensors,aiv_vec_time down to about 0.)
3.3 Counter mode (SetMaskCount): Do Vector instructions use Counter mode, rather than Normal mode manually computing main block/tail block masks?
3.4 Matmul AtomicAdd: When the Matmul result C needs to be added to the GM matrix D, is it set in IterateAll/GetTensorC to set enAtomic=1 fused accumulation? (can reduce about 12% cycles.)
3.5 Reduction instruction combination: When reducing a contiguous buffer to a scalar, is the BlockReduceSum + WholeReduceSum combination used, rather than multiple identical reduction instructions?

4. Memory

Detailed example:references/memory-prof.md

Checklist items:

4.1 UB Buffer fusion: Are the intermediate results of consecutive Vector operations (such as Exp → Abs) kept within UB, rather than round-tripped through GM?
4.2 L0C accumulation matrix multiplication：A1*B1 + A2*B2 + ... scenario, are Mmad results accumulated in place in CO1 (L0C), rather than written to GM each time and then summed in UB?
4.3 Small matrix resident in L1: When L1 cannot simultaneously hold the left and right matrices, is the smaller matrix kept resident in L1 after being loaded once, with only the larger matrix moved in a loop?
4.4 BT Buffer storing bias(separated architecture): Is the bias stored in the BT Buffer (C2) and fused in one step via Mmad , rather than doing Add separately in UB?
4.5 FP Buffer storing quantization parameters(separated architecture): Are the quantization parameters stored in the FP Buffer (C2PIPE2GM) and quantized inline via Fixpipe , rather than computed separately in UB?

5. Pipeline

Detailed example:references/pipeline-prof.md

Checklist items:

5.1 CopyIn/Compute/CopyOut paradigm: Is the operator divided into a three-stage pipeline, using TQue for inter-stage synchronization?
5.2 Double Buffer：InitBuffer is the buffer count set to 2, so that CopyIn/CopyOut overlaps with Compute? (precondition: loop count >= 2, and the transfer time is not negligible relative to the compute time.)
5.3 Asynchronous Iterate (MIX mode): In the Matmul MIX scenario, is Iterate<false>()/IterateAll<false>() used to avoid the AIC/AIV synchronization overhead of each iteration?

1.3 Output the check report

After checking all stages, output a summary in the following format:

## 优化排查报告

### 发现的问题（按预期收益排序）
1. [阶段 X.Y] <问题描述> — <预期收益>
2. [阶段 X.Y] <问题描述> — <预期收益>
...

### 已确认无问题
- [阶段 X.Y] <检查项描述>
...

### 优化计划
按预期收益从大到小排列，确定本轮优化的目标项。

Phase 2: Baseline — Save the current performance test results

Before optimization, the performance baseline must be saved so that an exact comparison can be made after optimization.

2.1 Check the existing performance verification results

Check whether the following already exist under csrc/ops/<op_name>/test/ :

<op_name>_perf_cases.jsonl — performance test cases
<op_name>_torch_npu_profiler_report.md — performance comparison report

2.2 Perform a performance evaluation when there are no results

If the above files do not exist or the results are outdated (for example, the code has been updated but the report has not been regenerated),MUST call the ascendc-operator-performance-eval skill to complete a full performance evaluation:

Read ascendc-operator-performance-eval SKILL.md
Generate performance cases (JSONL) according to its process, run the profiler, and generate a comparison report
Ensure the report containscustom operator vs benchmarkcomplete comparison data

2.3 Save the baseline snapshot

Back up the current performance report as a baseline file, named <op_name>_baseline_report.md, saved in the same test/ directory. This file will later be used to compare the optimization effect.

csrc/ops/<op_name>/test/
├── <op_name>_perf_cases.jsonl                 ← 性能用例（优化前后共用）
├── <op_name>_torch_npu_profiler_report.md     ← 当前报告（会被覆盖）
└── <op_name>_baseline_report.md               ← 基线快照（优化前的性能数据）

Phase 3: Optimization — Modify code after learning the knowledge

3.1 Learn operator development knowledge (MANDATORY)

Before modifying code, you MUST load the reference files of the ascendc-operator-code-gen skill, to ensure an accurate understanding of AscendC API, data movement, synchronization control, etc.

Load the following references as needed (located in ascendc-operator-code-gen/references/）：

Reference file	Purpose
`GUIDE.md`	Overview: template selection, code generation process
`data-copy-api.md`	DataCopy/DataCopyPad API details
`vector-compute-api.md`	Vector computation API details
`sync-control-api.md`	TQue/Pipe synchronization control
`resource-management-api.md`	TPipe/TBuf resource management
`basic-data-structures-api.md`	Basic structures such as LocalTensor/GlobalTensor
`kernel-constraints.md`	Kernel programming constraints and common pitfalls

Based on the optimization points found in Phase 1, selectively load relevant references. For example:

Optimize data movement → load data-copy-api.md
Optimize pipeline → load sync-control-api.md + resource-management-api.md
Optimize computation → load vector-compute-api.md

3.2 Formulate a modification plan

For each optimization point in the Phase 1 investigation report, formulate a specific code modification plan:

优化点 [X.Y]: <问题描述>
├── 修改文件: op_host / op_kernel / 两者
├── 修改内容: <具体代码变更描述>
├── 预期效果: <量化预期（如搬运时间减少 30%）>
└── 风险评估: <是否可能影响精度/是否需要修改 tiling>

3.3 Execute code modifications

Modify the code one by one according to the modification plan. Follow these rules when modifying:

MUST comply with the code-gen anti-pattern checklist：

NEVER Let FP16/BF16 directly participate in complex mathematical calculations; it must first be Cast to FP32
NEVER Pass an rvalue in EXEC_KERNEL_CMD
NEVER Use DataCopy for GM↔UB data movement; DataCopyPad must be used
NEVER Directly reuse the source tensor after ReduceSum/ReduceMax
NEVER Use in the kernel std::min/max/abs/sqrt/exp and other standard library functions
NEVER Pass repeatTime > 255 to the high-dimensional tiling API
NEVER Modify cmake/ or csrc/utils/ files under
NEVER Hardcode the number of cores or the UB size

3.4 Compile and install

After the modifications are complete, you must recompile and install:

source ${ASCEND_HOME_PATH}/set_env.sh
cd task/ascend-kernel
bash build.sh
pip install output/ascend_kernel*.whl --force-reinstall --no-deps

If compilation fails, enter the troubleshooting loop (at most 3 times).

Phase 4: Precision verification — Ensure the optimized functionality is correct

MANDATORY — After optimization, precision verification must be passed before performance comparison.

4.1 Invoke the precision evaluation skill

Read and execute the complete process in ascendc-operator-precision-eval SKILL.md:

Generate precision test cases (≥30 cases, covering all dtypes)
Run pytest precision tests
Generate a precision report (Markdown + JSON)
Display in the current conversationthe overview, failure summary, and key findings

4.2 Precision determination

Result	Handling
All passed	Proceed to Phase 5 performance verification
Partial failure	Analyze the cause of failure, roll back or fix the code, and re-enter Phase 3
Massive failure	Roll back all modifications in this round and re-analyze the optimization plan

Phase 5: Performance verification — Confirm the optimization effect

5.1 Run the performance test for the same case

Use the same performance case as in Phase 2（<op_name>_perf_cases.jsonl), invoke ascendc-operator-performance-eval skill to re-run the performance evaluation.

Key requirements:

MUST Use exactly the same perf_cases.jsonl as the baseline (no test cases may be added or removed)
MUST Generate a new <op_name>_torch_npu_profiler_report.md
MUST Present the comparison table, summary, and conclusion in the current conversation

5.2 Comparative Analysis

Compare the optimized performance data with the baseline saved in Phase 2:

## 优化效果对比

| Case | Shape | dtype | 基线 per-step(us) | 优化后 per-step(us) | 提升比 | 标杆 per-step(us) | vs 标杆 |
|------|-------|-------|-------------------|--------------------|---------|--------------------|---------|
| ...  | ...   | ...   | ...               | ...                | ...     | ...                | ...     |

### 汇总
- 平均提升: X%
- 最大提升: X%（Case Y）
- vs 标杆平均比值: 优化前 A → 优化后 B

5.3 Performance Determination

Result	Action
Performance improved(most cases are faster after optimization)	Optimization successful, output the final report
Performance not improved or regressed	Proceed to Phase 6 iterative optimization

Phase 6: Iterative Optimization (up to 3 rounds)

If Phase 5 determines that performance has not improved, proceed to iteration:

当前轮次: N (N ∈ {1, 2, 3})

├── N < 3: 回到 Phase 1，选择下一优先级优化点或调整方案
│   ├── 重新排查，分析上一轮修改为何未生效
│   ├── 选择新的优化点或调整上一轮的方案
│   └── 重复 Phase 3 → Phase 4 → Phase 5
│
└── N = 3: 停止迭代，输出最终报告（含所有轮次记录）

Iteration Record

Each iteration round must record:

### 第 N 轮优化
- 优化目标: [阶段 X.Y] <描述>
- 修改内容: <代码变更摘要>
- 精度结果: 通过 / 失败
- 性能结果: 提升 X% / 未提升 / 回退 Y%
- 决策: 保留本轮修改 / 回退 / 继续下一轮

Final Output

After all rounds are completed (either successfully improved or the 3-round limit is reached), output the final summary report.

Present in the current conversation (MANDATORY)

MUST Present the following content in the conversation,NEVER output only the file paths:

Optimization troubleshooting summary: all issues found and their handling status
Overall performance comparison table: three-way comparison of baseline → optimized → benchmark
Iteration history summary: optimization goal, result, and decision for each round
≥3 key conclusions: main bottlenecks, distribution of optimization gains, remaining optimization space, etc.
File paths last: report and code file paths

File artifacts

csrc/ops/<op_name>/test/
├── <op_name>_perf_cases.jsonl                 ← 性能用例
├── <op_name>_baseline_report.md               ← 优化前基线
├── <op_name>_torch_npu_profiler_report.md     ← 优化后最终性能报告
├── <op_name>_precision_report.md              ← 精度验证报告
└── <op_name>_optim_summary.md                 ← 优化迭代汇总报告（新增）

csrc/ops/<op_name>/
├── op_host/<op_name>.cpp                      ← 优化后的 host 代码
└── op_kernel/<op_name>.cpp                    ← 优化后的 kernel 代码

Optimization iteration summary report structure

<op_name>_optim_summary.md Must include:

# <op_name> 性能优化报告

## 排查发现
（Phase 1 的排查报告内容）

## 优化前基线
（Phase 2 的性能数据摘要）

## 迭代历史

### 第 1 轮
- 优化目标: ...
- 代码修改: ...
- 精度结果: ...
- 性能结果: ...

### 第 N 轮
...

## 最终性能对比
（优化前 vs 优化后 vs 标杆 三方对比表）

## 结论
（≥3 条关键发现）

Checklist (assistant self-check)

Phase 1: Troubleshooting

Have read the operator design document (design.md)
Have read the complete op_host + op_kernel source code
Have loaded reference stage by stage and checked item by item
Have output the troubleshooting report, with optimization points sorted by expected benefit

Phase 2: Baseline

Have confirmed or generated performance test cases (JSONL)
Have confirmed or run performance evaluation (custom vs benchmark)
Have saved the baseline snapshot (_baseline_report.md）

Phase 3: Optimization

Have loaded code-gen reference (required reading before modification)
Code modifications comply with the anti-pattern list
Compilation and installation succeeded

Phase 4: Accuracy

Have followed the ascendc-operator-precision-eval process to complete accuracy verification
Accuracy verification passed (all or most test cases PASS)

Phase 5: Performance

Used the same perf_cases.jsonl as the baseline
Have presented the performance comparison data in the conversation
Have determined whether improvement occurred

Phase 6: Iteration

Iterations do not exceed 3 rounds
Each round has a record (goal, modifications, accuracy, performance, decision)
Have output the final summary report (_optim_summary.md）

Output

Already presented in the current conversationtroubleshooting summary, performance comparison, iteration history, ≥3 conclusions
NEVER Output only the file paths

About this skill

Phase 1: Troubleshoot — discover optimization points

1.1 Study the operator design document

1.2 Troubleshoot stage by stage

1. Tiling

2. Data movement

3. API usage

4. Memory

5. Pipeline

1.3 Output the check report

Phase 2: Baseline — Save the current performance test results

2.1 Check the existing performance verification results

2.2 Perform a performance evaluation when there are no results

2.3 Save the baseline snapshot

Phase 3: Optimization — Modify code after learning the knowledge

3.1 Learn operator development knowledge (MANDATORY)

3.2 Formulate a modification plan

3.3 Execute code modifications

3.4 Compile and install

Phase 4: Precision verification — Ensure the optimized functionality is correct

4.1 Invoke the precision evaluation skill

4.2 Precision determination

Phase 5: Performance verification — Confirm the optimization effect

5.1 Run the performance test for the same case

5.2 Comparative Analysis

5.3 Performance Determination

Phase 6: Iterative Optimization (up to 3 rounds)

Iteration Record

Final Output

Present in the current conversation (MANDATORY)

File artifacts

Optimization iteration summary report structure

Checklist (assistant self-check)

Phase 1: Troubleshooting

Phase 2: Baseline

Phase 3: Optimization

Phase 4: Accuracy

Phase 5: Performance

Phase 6: Iteration

Output