ascendc-operator-performance-optim

Ascend C operator performance optimization (investigate → modify → verify loop)

This skill not only investigates performance issues, but is also responsible for modifying code and verifying the optimization effect. The complete process is:

Phase 1: 排查 — 审查代码 + 学习设计文档，发现优化点
Phase 2: 基线 — 保存当前性能测试结果（自定义算子 vs 标杆）
Phase 3: 优化 — 学习 code-gen 知识后修改算子代码
Phase 4: 精度 — 精度验证（确保优化后功能正确）
Phase 5: 性能 — 同 case 性能对比（优化后 vs 标杆）
Phase 6: 迭代 — 未提升则继续优化，最多 3 轮

Phase 1: Investigate — discover optimization points

1.1 Study the operator design document

MANDATORY — before investigating, you must first understand the operator design：

Read ascend-kernel/csrc/ops/<op_name>/design.md(if it exists), and extract:
- Operator type (elementwise / row processing / Cube)
- Tiling strategy (inter-core partitioning / intra-core partitioning)
- UB space allocation scheme
- Computation logic and data flow
Read op_host/<op_name>.cpp and op_kernel/<op_name>.cpp all source code

1.2 Investigate stage by stage

Review the operator code stage by stage in the following order. For each stage, load the corresponding reference file and check the code item by item.

- [ ] 1. Tiling    — 数据在多核与 L2Cache 间的切分策略
- [ ] 2. 搬运      — DataCopy 的带宽利用率
- [ ] 3. API 使用  — Ascend C API 的高效用法
- [ ] 4. 内存      — 数据在存储层级中的放置策略
- [ ] 5. 流水      — CopyIn / Compute / CopyOut 的重叠执行

Each stage has an independent reference file; during investigation,load only the current stage'sfile:

1. Tiling

Detailed example:references/tiling-prof.md

Investigation items:

1.1 Multi-core partitioning：blockDim Is it set to the number of hardware cores?
- Coupled architecture:GetCoreNumAiv() or GetCoreNumAic()
- Separated architecture Vector operator: number of AIV cores (e.g. 40)
- Separated architecture Cube operator: number of AIC cores (e.g. 20)
- Separated architecture MIX operator: number of physical core groups (e.g. 20 = 40 AIV / 2), must not exceed the number of physical cores
1.2 L2Cache partitioning: when 输入 + 输出 > L2Cache 容量 , is the data partitioned by L2Cache size, with all cores cooperatively processing the same block before switching to the next block?
1.3 Inter-core load balancing: after L2Cache partitioning, is the tail block alternately assigned across passes, to avoid certain cores always having the tail?

2. Data movement

Detailed example:references/data-copy-prof.md

Investigation items:

2.1 Single transfer amount >= 16 KB: does each DataCopy transfer at least 16 KB? Below this value, bandwidth utilization drops significantly.
2.2 GM address 512B alignment: is the GM start address 512-byte aligned? (On the Atlas A2 series, 32B alignment has up to 30% lower bandwidth than 512B alignment.)
2.3 Use stride parameters instead of a for loop: for strided transfers, is DataCopyParams (blockCount/blockLen/srcStride/dstStride) issued in one go, rather than using a for loop to transfer row by row?

3. API usage

Detailed example:references/api-usage-prof.md

Investigation items:

3.1 Create TPipe outside the kernel class：TPipe Is it created in the kernel entry function and passed in as a pointer to the class? (An in-class TPipe prevents Scalar constant folding, increasing scalar_time by about 17%.)
3.2 Pure data movement operators use TQueBind: for operators without Vector computation, is TQueBind<VECIN, VECOUT> used instead of separate TQue<VECIN> + TQue<VECOUT>? (Eliminate redundant DataCopy between LocalTensors,aiv_vec_time reduced to about 0.)
3.3 Counter mode (SetMaskCount): Do Vector instructions use Counter mode, rather than Normal mode manually computing the main block/tail block mask?
3.4 Matmul AtomicAdd: When the Matmul result C needs to be added to the GM matrix D, is IterateAll/GetTensorC set to enAtomic=1 fused accumulation? (Can reduce about 12% cycles.)
3.5 Reduction instruction combination: When reducing a continuous buffer to a scalar, is the BlockReduceSum + WholeReduceSum combination used, rather than multiple identical reduction instructions?

4. Memory

Detailed example:references/memory-prof.md

Check items:

4.1 UB Buffer fusion: Are the intermediate results of consecutive Vector operations (such as Exp → Abs) kept in UB, rather than round-tripped through GM?
4.2 L0C accumulation matrix multiplication：A1*B1 + A2*B2 + ... In this scenario, are the Mmad results accumulated in place in CO1 (L0C), rather than written to GM one by one and then summed in UB?
4.3 Small matrix resident in L1: When L1 cannot hold both the left and right matrices at the same time, is the smaller matrix after being loaded once kept resident in L1, with only the larger matrix moved in a loop?
4.4 BT Buffer stores bias(separated architecture): Is the bias stored in BT Buffer (C2) and fused in one step via Mmad , rather than doing Add separately in UB?
4.5 FP Buffer stores quantization parameters(separated architecture): Are the quantization parameters stored in FP Buffer (C2PIPE2GM) and quantized on the fly via Fixpipe , rather than computed separately in UB?

5. Pipeline

Detailed example:references/pipeline-prof.md

Check items:

5.1 CopyIn/Compute/CopyOut paradigm: Is the operator divided into a three-stage pipeline, using TQue for inter-stage synchronization?
5.2 Double Buffer：InitBuffer Is the buffer count set to 2, so that CopyIn/CopyOut and Compute execute in an overlapping manner? (Prerequisite: the number of loop iterations >= 2, and the copy time is non-negligible relative to the compute time.)
5.3 Asynchronous Iterate (MIX mode): In the Matmul MIX scenario, is Iterate<false>()/IterateAll<false>() used to avoid the AIC/AIV synchronization overhead of each iteration?

1.3 Output the check report

After checking all stages, output a summary in the following format:

## 优化排查报告

### 发现的问题（按预期收益排序）
1. [阶段 X.Y] <问题描述> — <预期收益>
2. [阶段 X.Y] <问题描述> — <预期收益>
...

### 已确认无问题
- [阶段 X.Y] <检查项描述>
...

### 优化计划
按预期收益从大到小排列，确定本轮优化的目标项。

Phase 2: Baseline — Save the current performance test results

Before optimization, the performance baseline must be saved so that an accurate comparison can be made after optimization.

2.1 Check existing performance verification results

Check whether the following already exists under csrc/ops/<op_name>/test/ :

<op_name>_perf_cases.jsonl — performance test case
<op_name>_torch_npu_profiler_report.md — performance comparison report

2.2 Perform performance evaluation when there are no results

If the above files do not exist or the results are outdated (for example, the code has been updated but the report has not been regenerated),MUST call the ascendc-operator-performance-eval skill to complete a full performance evaluation:

Read ascendc-operator-performance-eval SKILL.md
, follow its process to generate performance cases (JSONL), run the profiler, and generate a comparison report
Ensure the report includescustom operator vs benchmarkcomplete comparison data

2.3 Save the baseline snapshot

Back up the current performance report as a baseline file, named <op_name>_baseline_report.md, saved in the same test/ directory. This file will later be used to compare the optimization effect.

csrc/ops/<op_name>/test/
├── <op_name>_perf_cases.jsonl                 ← 性能用例（优化前后共用）
├── <op_name>_torch_npu_profiler_report.md     ← 当前报告（会被覆盖）
└── <op_name>_baseline_report.md               ← 基线快照（优化前的性能数据）

Phase 3: Optimization — Modify code after learning knowledge

3.1 Learn operator development knowledge (MANDATORY)

Before modifying code, MUST load the reference files of the ascendc-operator-code-gen skill, to ensure an accurate understanding of AscendC APIs, data movement, synchronization control, etc.

Load the following references as needed (located in ascendc-operator-code-gen/references/）：

Reference file	Purpose
`GUIDE.md`	Overview: template selection, code generation process
`data-copy-api.md`	Detailed explanation of the DataCopy/DataCopyPad API
`vector-compute-api.md`	Detailed explanation of the Vector computation API
`sync-control-api.md`	TQue/Pipe synchronization control
`resource-management-api.md`	TPipe/TBuf resource management
`basic-data-structures-api.md`	Basic structures such as LocalTensor/GlobalTensor
`kernel-constraints.md`	Kernel programming constraints and common pitfalls

Based on the optimization points found in Phase 1, selectively load the relevant references. For example:

Optimize data movement → load data-copy-api.md
Optimize pipeline → load sync-control-api.md + resource-management-api.md
Optimize computation → load vector-compute-api.md

3.2 Formulate a modification plan

For each optimization point in the Phase 1 investigation report, formulate a specific code modification plan:

优化点 [X.Y]: <问题描述>
├── 修改文件: op_host / op_kernel / 两者
├── 修改内容: <具体代码变更描述>
├── 预期效果: <量化预期（如搬运时间减少 30%）>
└── 风险评估: <是否可能影响精度/是否需要修改 tiling>

3.3 Execute code modifications

Modify the code one by one according to the modification plan. Observe the following rules when modifying:

MUST comply with the code-gen anti-pattern checklist：

NEVER Let FP16/BF16 directly participate in complex mathematical computations; they must first be Cast to FP32
NEVER Pass an rvalue in EXEC_KERNEL_CMD
NEVER Use DataCopy for GM↔UB movement; must use DataCopyPad
NEVER Directly reuse the source tensor after ReduceSum/ReduceMax
NEVER Use std::min/max/abs/sqrt/exp and other standard library functions in the kernel
NEVER Pass repeatTime > 255 to a high-dimensional tiling API
NEVER Modify cmake/ or csrc/utils/ files under
NEVER Hard-code the number of cores or the UB size

3.4 Compile and install

After modification is complete, you must recompile and install:

source ${ASCEND_HOME_PATH}/set_env.sh
cd task/ascend-kernel
bash build.sh
pip install output/ascend_kernel*.whl --force-reinstall --no-deps

If compilation fails, enter the troubleshooting loop (at most 3 times).

Phase 4: Precision verification — Ensure correct functionality after optimization

MANDATORY — After optimization, precision verification must be passed before performance comparison.

4.1 Invoke the precision evaluation skill

Read and execute the complete process of ascendc-operator-precision-eval SKILL.md:

Generate precision test cases (≥30 cases, covering all dtypes)
Run pytest precision tests
Generate a precision report (Markdown + JSON)
Display in the current conversation theoverview, failure summary, and key findings

4.2 Precision determination

Result	Handling
All pass	Proceed to Phase 5 performance verification
Partial failure	Analyze the cause of failure, roll back or fix the code, and re-enter Phase 3
Massive failure	Roll back all modifications in this round and re-analyze the optimization plan

Phase 5: Performance verification — Confirm the optimization effect

5.1 Run the performance test for the same case

Use the same performance cases as in Phase 2（<op_name>_perf_cases.jsonl), invoke ascendc-operator-performance-eval the skill to re-run the performance evaluation.

Key requirements:

MUST Use exactly the same perf_cases.jsonl as the baseline (no test cases may be added or removed)
MUST Generate a new <op_name>_torch_npu_profiler_report.md
MUST Display the comparison table, summary, and conclusions in the current conversation

5.2 Comparative Analysis

Compare the optimized performance data with the baseline saved in Phase 2:

## 优化效果对比

| Case | Shape | dtype | 基线 per-step(us) | 优化后 per-step(us) | 提升比 | 标杆 per-step(us) | vs 标杆 |
|------|-------|-------|-------------------|--------------------|---------|--------------------|---------|
| ...  | ...   | ...   | ...               | ...                | ...     | ...                | ...     |

### 汇总
- 平均提升: X%
- 最大提升: X%（Case Y）
- vs 标杆平均比值: 优化前 A → 优化后 B

5.3 Performance Determination

Result	Action
Performance improved(most cases are faster after optimization)	Optimization successful, output the final report
Performance not improved or regressed	Proceed to Phase 6 iterative optimization

Phase 6: Iterative Optimization (up to 3 rounds)

If Phase 5 determines that performance has not improved, proceed to iteration:

当前轮次: N (N ∈ {1, 2, 3})

├── N < 3: 回到 Phase 1，选择下一优先级优化点或调整方案
│   ├── 重新排查，分析上一轮修改为何未生效
│   ├── 选择新的优化点或调整上一轮的方案
│   └── 重复 Phase 3 → Phase 4 → Phase 5
│
└── N = 3: 停止迭代，输出最终报告（含所有轮次记录）

Iteration Record

Each iteration round must record:

### 第 N 轮优化
- 优化目标: [阶段 X.Y] <描述>
- 修改内容: <代码变更摘要>
- 精度结果: 通过 / 失败
- 性能结果: 提升 X% / 未提升 / 回退 Y%
- 决策: 保留本轮修改 / 回退 / 继续下一轮

Final Output

After all rounds are completed (either successfully improved or the 3-round limit is reached), output the final summary report.

Display in the current conversation (MANDATORY)

MUST Display the following content in the conversation,NEVER output only the file paths:

Optimization investigation summary: all issues found and their handling status
Overall performance comparison table: three-way comparison of baseline → optimized → benchmark
Iteration history summary: optimization goal, result, and decision for each round
≥3 key conclusions: main bottlenecks, distribution of optimization gains, remaining optimization potential, etc.
File paths at the end: report and code file paths

File Artifacts

csrc/ops/<op_name>/test/
├── <op_name>_perf_cases.jsonl                 ← 性能用例
├── <op_name>_baseline_report.md               ← 优化前基线
├── <op_name>_torch_npu_profiler_report.md     ← 优化后最终性能报告
├── <op_name>_precision_report.md              ← 精度验证报告
└── <op_name>_optim_summary.md                 ← 优化迭代汇总报告（新增）

csrc/ops/<op_name>/
├── op_host/<op_name>.cpp                      ← 优化后的 host 代码
└── op_kernel/<op_name>.cpp                    ← 优化后的 kernel 代码

Optimization iteration summary report structure

<op_name>_optim_summary.md Must include:

# <op_name> 性能优化报告

## 排查发现
（Phase 1 的排查报告内容）

## 优化前基线
（Phase 2 的性能数据摘要）

## 迭代历史

### 第 1 轮
- 优化目标: ...
- 代码修改: ...
- 精度结果: ...
- 性能结果: ...

### 第 N 轮
...

## 最终性能对比
（优化前 vs 优化后 vs 标杆 三方对比表）

## 结论
（≥3 条关键发现）

Checklist (assistant self-check)

Phase 1: Investigation

Have read the operator design document (design.md)
Have read the complete op_host + op_kernel source code
Have loaded the reference stage by stage and investigated item by item
Have output the investigation report, with optimization points sorted by expected gain

Phase 2: Baseline

Have confirmed or generated performance test cases (JSONL)
Have confirmed or run performance evaluation (custom vs benchmark)
Have saved the baseline snapshot (_baseline_report.md）

Phase 3: Optimization

Have loaded the code-gen reference (required reading before modification)
Code modifications comply with the anti-pattern checklist
Compilation and installation succeeded

Phase 4: Accuracy

Have followed the ascendc-operator-precision-eval process to complete accuracy verification
Accuracy verification passed (all or most test cases PASS)

Phase 5: Performance

Used the same perf_cases.jsonl as the baseline
Have displayed the performance comparison data in the conversation
Have determined whether performance improved

Phase 6: Iteration

No more than 3 iteration rounds
Each round has a record (goal, modifications, accuracy, performance, decision)
Have output the final summary report (_optim_summary.md）

Output

Displayed in the current conversationinvestigation summary, performance comparison, iteration history, ≥3 conclusions
NEVER output only the file paths

About this skill

Phase 1: Investigate — discover optimization points

1.1 Study the operator design document

1.2 Investigate stage by stage

1. Tiling

2. Data movement

3. API usage

4. Memory

5. Pipeline

1.3 Output the check report

Phase 2: Baseline — Save the current performance test results

2.1 Check existing performance verification results

2.2 Perform performance evaluation when there are no results

2.3 Save the baseline snapshot

Phase 3: Optimization — Modify code after learning knowledge

3.1 Learn operator development knowledge (MANDATORY)

3.2 Formulate a modification plan

3.3 Execute code modifications

3.4 Compile and install

Phase 4: Precision verification — Ensure correct functionality after optimization

4.1 Invoke the precision evaluation skill

4.2 Precision determination

Phase 5: Performance verification — Confirm the optimization effect

5.1 Run the performance test for the same case

5.2 Comparative Analysis

5.3 Performance Determination

Phase 6: Iterative Optimization (up to 3 rounds)

Iteration Record

Final Output

Display in the current conversation (MANDATORY)

File Artifacts

Optimization iteration summary report structure

Checklist (assistant self-check)

Phase 1: Investigation

Phase 2: Baseline

Phase 3: Optimization

Phase 4: Accuracy

Phase 5: Performance

Phase 6: Iteration

Output