external-gitcode-ascend-catlass-operator-performance-optim

Catlass Operator Performance Tuning

Core Workflow

阅读优化指南 → 获取基准数据 → 修改 Tiling 配置 → 重新编译运行 → profiler 双路径采集
    → 输出性能对比报告（落盘 + 聊天展示）→ 对比迭代 → 确定最佳配置

Prerequisites

Check Item	Description
Project Directory	Exists OPS_PROJECT_ROOT, and under it there is already `catlass/docs/1_Practice/10_matmul_optimization.md`
Operator Status	The operator can already be compiled and run
Performance Data	The baseline is preferably `csrc/ops/<op>/test/` under ascendc-operator-performance-eval collected according to the `<op>_torch_npu_profiler_report.md`specification (or the equivalent Markdown generated by this project's `benchmark_*_torch_npu_profiler.py` ).ascendc-operator-precision-eval is for accuracy only, not used as a performance baseline

When conditions are unclearAsk follow-up questions tothe user.

Tuning Principles

Tuning StrategiesFollow the official catlass optimization guide
Each modificationadjusts only one variable, so as to accurately evaluate the optimization effect
Record all attempted configurations and the corresponding performance data

NEVER / ALWAYS

NEVER: randomly changing tiling parameters without reading the catlass optimization guide; modifying multiple variables at once; ignoring hardware resource limits; not recompiling after modifying the code;when the NPU and profiler are available, only modifying the code without producing a comparison report

ALWAYS: read the catlass optimization guide before starting; record the configuration and performance data at each iteration; roll back the configuration when performance decreases; after modifying the code, use ascendc-operator-compile-debug to recompile;after each meaningful performance optimization is completed, produce and present a performance comparison report according to §6 (written to disk + chat interface)

Steps

1. Read the optimization guide

Under OPS_PROJECT_ROOT , read:<OPS_PROJECT_ROOT>/catlass/docs/1_Practice/10_matmul_optimization.md

Understand the tunable parameters: TileShape, DispatchPolicy, Swizzle, etc.Be sure to read the documentation and conduct optimization attempts according to its recommendations, rather than randomly modifying parameters。

2. Obtain baseline data

The iteration baseline is taken from the report comparison table produced by ascendc-operator-performance-eval(or the equivalent torch_npu.profiler script in the operator directory), specifically the elapsed time of each case and the native/custom ratio. Before tuning, complete the collection in the same csrc/ops/<op>/test/ directory and obtain the first version of the report as the "before optimization" reference.

3. Modify the Tiling configuration

According to the recommendations in the optimization guide, modify the operator's tiling shape configuration:

Modification Item	Description
TileShape size	Adjust the L1/L0 layer TileShape
DispatchPolicy	Select another scheduling policy
Swizzle policy	Try modifying the data movement Swizzle

4. Recompile and collect

After modifying the code, you must recompile (using ascendc-operator-compile-debug), then run the operator to collect performance data and compare it with the baseline:

Performance improved → record the configuration and continue trying other optimization recommendations
Performance decreased → roll back the configuration and try other approaches
Performance stable → fine-tune around the current configuration

5. Iterate and determine the best configuration

Within a reasonable number of iterations, find the tiling configuration with the best performance. Record the detailed parameters of the best configuration and the corresponding performance metrics.

6. Performance Comparison Report Delivery (Mandatory)

Each time a round of submittable performance optimization is completed(or a tuning conclusion is delivered to the user), NPU and torch_npu.profiler are available under the premise MUST execute:

No.	Requirement
6.1	According to ascendc-operator-performance-eval agreement:warmup=5、active=5，custom operator vs benchmarkboth paths are collected on NPU ; the benchmark may be `torch.matmul` / `torch_npu` an equivalent API or a concatenation of small operators (consistent with the eval skill).
6.2	In `csrc/ops/<op_name>/test/` write to disk Markdown performance comparison report(the file name just needs to be consistent with the engineering script, e.g. `<op>_torch_npu_profiler_report.md`). The report must contain comparison tables such as Case / Shape / custom per-step / benchmark per-step / ratio , and note the case JSONL, trace root directory, and schedule.
6.3	Show in the current conversation: at least paste the main body of the comparison table or the row corresponding to the target shape, the report and the JSONL'sfull paths, and use 1 to 3 sentencesto explain the conclusion (e.g., which is faster, the native/custom ratio for key cases, whether expectations are met).Must notwrite only "optimized" without attaching data.
6.4	If the environment is unavailable (no NPU, profiler failure):honestly explain the reason, list the configurations already changed and the collection commands suggested for the user to run locally,do not fabricate reports。

6.5 Before / After Optimization Comparison (mandatory relative to the previous Kernel version)

When this round of tasks includes the conclusion of "performance change relative to before optimization" (rather than only comparing with torch.matmul the benchmark),MUST additionally satisfy:

No.	Requirement
6.5.1	Before modifying the Kernel/tiling:under the same CANN / driver / clockconditions, run a complete torch_npu.profileronce with the current binary (same agreement as §6.1), write to disk the before optimizationreport, suggested naming `<op>_torch_npu_profiler_report_PRE.md`(or `*_baseline.md`), and keep the corresponding trace directory for reference.
6.5.2	After modifying the code and recompiling/installing: run another round of profiler, write to disk the after optimizationreport, suggested naming `<op>_torch_npu_profiler_report_POST.md`。
6.5.3	Merged delivery: generate before optimization vs after optimization comparison table, containing at least Case / Shape / engineering custom per-step (before) / (after) / engineering Δ%; the benchmark column may be taken from the after-optimization report. It is allowed to use the operator `test/` merge script (e.g. `merge_profiler_before_after.py`) or an equivalent table.
6.5.4	Show in the current conversationThis comparison table (at least covering the shapes the user named), andtruthfully statethe cases where some cases regress(Δ%>0); it is forbidden to cherry-pick only the cases that got faster while hiding the regressions.

Note: it coexists with the torch.matmul dual-path report (§6.2)coexists: the former answers "how far from the framework", while §6.5 answers "how much better this Kernel change is than before the change".

Note: this step aligns with catlass-operator-dev Phase 6 "Show performance summary in chat"; this skill emphasizes that after tuning iterations, a reviewable Markdown + chat summary must likewise be delivered。

Quality verification

Have read the catlass optimization guide, not arbitrary modifications
Only one variable adjusted per iteration (or a single combined variable with the exception reason already explained in the report)
All attempted configurations and performance data recorded
Best configuration determined and its parameters and metrics recorded
Performance comparison Markdown generated and written to disk, and the table and conclusion shown in the current conversation(satisfies §6; if the environment does not allow it, this has been explained)
If a "before/after optimization" effect is claimed: PRE/POST two profiler reports and the merged comparison table(§6.5) have been written to disk, and a table containing Δ% has been shown in the conversation
If asingle target shape tiling sweep was performed: test/ has been updated with the TILING_EXPLORATION_*.md and TILING_SEARCH.md summary, and no minimal probe kernel has been left behind

References

Document	Purpose
`<ASCEND_KERNEL_ROOT>/catlass/docs/1_Practice/10_matmul_optimization.md`	Tuning strategy follows this
ascendc-operator-performance-eval	Performance baseline and profiler comparison workflow, JSONL specification
ascendc-operator-compile-debug	Recompile and install the whl after modifying code

Continuous tiling search (recommended)

When the target performance has not yet been reached, csrc/ops/<op>/test/ should maintain TILING_SEARCH.md(or an equivalent document): list by 10_matmul_optimization.md the verified ineffective/regressed configurations and the to try items (Swizzle offset, Preload, Split-K, TLA, etc.), checking off each item PRE/POST after comparison; avoid repeating the same pitfalls.

Tiling exploration for a single target shape (notes consolidation)

When the usercares about only a few large shapes(e.g. a single M×N×K) and wants, relative to the Catlass example / documented default tiling , to find a better combination of L1/L0/Swizzle :

Requirement	Description
Probe notes	In `csrc/ops/<op>/test/` add or maintain `TILING_EXPLORATION_<brief>.md`(e.g. `TILING_EXPLORATION_1280x16384x4096.md`), and in `TILING_SEARCH.md` 's "verified conclusions", a one-line summary + link, so as to avoid mixing it up with full-shape tuning in one mess.
Table fields	Suggested columns:Date、Label、L1(M,N,K)、L0(if different from default),Swizzle、Engineering custom per-step (μs)、Baseline (μs)、Relative to default Δ% or conclusion(best / fallback / pending retest). For each new probe append one row, do not overwrite history.
Isolation probe (replace kernel for the entire file)	To run only one tile, you may temporarilychange `op_kernel` to a minimal kernel with "instantiate the full problem as a single instance";before the same round of tasks ends, you must restorewith shape dispatchproduction version, and compile and install again;forbiddento treat the minimal kernel used for probing as the final deliverable.
Relationship with §6	Before formally adopting a configuration, you should still run full JSONL or the user-specified case set for PRE/POST(§6.5) and save the report to disk; the single-shape table only records the search process, and does not replace full regression.