catlass-operator-performance-optim

Catlass Operator Performance Tuning

Core Workflow

阅读优化指南 → 获取基准数据 → 修改 Tiling 配置 → 重新编译运行 → profiler 双路径采集
    → 输出性能对比报告（落盘 + 聊天展示）→ 对比迭代 → 确定最佳配置

Prerequisites

Check Item	Description
Project Directory	Exists OPS_PROJECT_ROOT, and under it there is already `catlass/docs/1_Practice/10_matmul_optimization.md`
Operator Status	The operator can already be compiled and run
Performance Data	The baseline is preferably `csrc/ops/<op>/test/` collected under ascendc-operator-performance-eval according to the `<op>_torch_npu_profiler_report.md`specification (or the equivalent Markdown generated by this project's `benchmark_*_torch_npu_profiler.py` ).ascendc-operator-precision-eval is only for accuracy, not used as a performance baseline

When conditions are unclearask follow-up questions tothe user.

Tuning Principles

Tuning StrategySubject to the catlass official optimization guide
Each modificationadjusts only one variable, so as to accurately evaluate the optimization effect
Record all attempted configurations and their corresponding performance data

NEVER / ALWAYS

NEVER: Arbitrarily changing tiling parameters without reading the catlass optimization guide; modifying multiple variables at once; ignoring hardware resource limits; not recompiling after modifying code;when the NPU and profiler are available, only modifying code without producing a comparison report

ALWAYS: Read the catlass optimization guide before starting; record the configuration and performance data for each iteration; roll back the configuration when performance decreases; after modifying code, use ascendc-operator-compile-debug to recompile;after each meaningful performance optimization is completed, produce and present a performance comparison report according to §6 (write to disk + chat interface)

Steps

1. Read the optimization guide

Under OPS_PROJECT_ROOT read:<OPS_PROJECT_ROOT>/catlass/docs/1_Practice/10_matmul_optimization.md

Understand the tunable parameters: TileShape, DispatchPolicy, Swizzle, etc.Be sure to read the documentation and carry out optimization attempts according to its suggestions, rather than arbitrarily modifying parameters。

2. Obtain baseline data

The iteration baseline is taken from ascendc-operator-performance-eval(or the equivalent torch_npu.profiler script in the operator directory) in the report comparison table: the elapsed time of each case and the native/custom ratio. Before tuning, complete collection in the same csrc/ops/<op>/test/ directory and obtain the first version of the report, as the "before optimization" reference.

3. Modify Tiling Configuration

According to the suggestions in the optimization guide, modify the operator's tiling shape configuration:

Modification Item	Description
TileShape size	Adjust the L1/L0 layer TileShape
DispatchPolicy	Select another scheduling policy
Swizzle policy	Try modifying the data movement Swizzle

4. Recompile and Collect

After modifying code, you must recompile (using ascendc-operator-compile-debug), then run the operator to collect performance data and compare it with the baseline:

Performance improvement → record the configuration and continue trying other optimization suggestions
Performance decrease → roll back the configuration and try other approaches
Performance stable → fine-tune around the current configuration

5. Iterate and determine the best configuration

Within a reasonable number of iterations, find the tiling configuration with the best performance. Record the detailed parameters of the best configuration and the corresponding performance metrics.

6. Performance comparison report delivery (mandatory)

Each time a round of submittable performance optimization is completed(or a tuning conclusion is delivered to the user), under the premise that NPU and torch_npu.profiler are available , execute MUST :

No.	Requirement
6.1	As per ascendc-operator-performance-eval the agreement:warmup=5、active=5，custom operator vs benchmarkboth paths are collected on NPU ; the benchmark may be `torch.matmul` / `torch_npu` an equivalent API or a combination of small operators (consistent with the eval skill).
6.2	Write to disk at `csrc/ops/<op_name>/test/` a Markdown performance comparison report(the file name just needs to be consistent with the engineering script, for example `<op>_torch_npu_profiler_report.md`). The report must contain Case / Shape / custom per-step / benchmark per-step / ratio and other comparison tables, and note the case JSONL, trace root directory, and schedule.
6.3	Display in the current conversation: show at least the main body of the comparison table or the row corresponding to the target shape, thefull pathsof the report and JSONL, and use 1 to 3 sentencesto explain the conclusion (for example: which is faster, the native/custom ratio for key cases, whether expectations are met).Must notwrite only "optimized" without accompanying data.
6.4	If the environment is unavailable (no NPU, profiler failure):honestly explain the reason, list the configurations that have been changed and the collection commands suggested for the user to execute locally,do not fabricate a report。

6.5 Before / after optimization comparison (mandatory relative to the previous Kernel version)

When the current round of tasks includes "performance change relative to before optimization" conclusions (rather than only comparison with the torch.matmul benchmark),MUST additionally satisfy:

No.	Requirement
6.5.1	Before changing the Kernel/tiling: underthe same CANN / driver / clockconditions, run a full torch_npu.profileronce with the current binary (same agreement as §6.1), write to disk the before-optimizationreport, recommended naming `<op>_torch_npu_profiler_report_PRE.md`(or `*_baseline.md`), and keep the corresponding trace directory for reference.
6.5.2	After changing the code and recompiling and installing: run another round of profiler, write to disk the after-optimizationreport, recommended naming `<op>_torch_npu_profiler_report_POST.md`。
6.5.3	Merged delivery: generate a before-optimization vs after-optimization comparison table, containing at least Case / Shape / engineered custom per-step (before) / (after) / engineered Δ%; the benchmark column may be taken from the after-optimization report. It is allowed to use the merge script under the operator `test/` (such as `merge_profiler_before_after.py`) or an equivalent table.
6.5.4	Display in the current conversationThis comparison table (at least covering the shapes named by the user), andtruthfully explaincases where some regress(Δ%>0); it is forbidden to cherry-pick only the cases that got faster while concealing regressions.

Note: this coexists with the dual-path report against the benchmark torch.matmul (§6.2)coexists: the former answers "how far from the framework", while §6.5 answers "how much better is this Kernel change than before the change".

Note: this step aligns with catlass-operator-dev Phase 6 "Show performance summary in chat"; this skill emphasizes that after tuning iterations, a reviewable Markdown + chat summary must also be delivered。

Quality verification

Has read the catlass optimization guide, not made arbitrary changes
Only one variable adjusted per iteration (or the reason for the exception of merging variables in a single iteration has been explained in the report)
All attempted configurations and performance data have been recorded
The best configuration has been determined and its parameters and metrics recorded
A performance comparison Markdown has been generated and persisted to disk, and the table and conclusion have been shown in the current conversation(satisfies §6; if the environment does not allow it, this has been explained)
If a "before/after optimization" effect has been claimed: both profiler reports have been persisted to disk PRE/POST and the merged comparison table(§6.5), and the table containing Δ% has been shown in the conversation
If asingle-target shape tiling sweep was performed: test/ has been updated with TILING_EXPLORATION_*.md and TILING_SEARCH.md summary, and no minimalist probe kernel has been left behind

Reference materials

Document	Purpose
`<ASCEND_KERNEL_ROOT>/catlass/docs/1_Practice/10_matmul_optimization.md`	Tuning strategy follows this
ascendc-operator-performance-eval	Performance baseline and profiler comparison workflow, JSONL specification
ascendc-operator-compile-debug	Recompile and install the whl after modifying code

Continuous tiling search (recommended)

When the target performance has not yet been reached, csrc/ops/<op>/test/ maintain TILING_SEARCH.md(or an equivalent document): list by 10_matmul_optimization.md the verified ineffective/regressed configurations and to-try items (Swizzle offset, Preload, Split-K, TLA, etc.), check off each item only after PRE/POST comparison; avoid repeating the same pitfalls.

Tiling exploration for a single target shape (consolidated in notes)

When the useronly cares about a few large shapes(e.g., a single M×N×K) and wants to find a better Catlass example / documented default tiling combination compared to L1/L0/Swizzle :

Requirement	Description
Probe notes	In `csrc/ops/<op>/test/` add or maintain `TILING_EXPLORATION_<brief>.md`(e.g., `TILING_EXPLORATION_1280x16384x4096.md`), and in `TILING_SEARCH.md` 's "verified conclusions" add a one-line summary + link, to avoid mixing it all together with full-shape tuning.
Table fields	Suggested columns:Date、Label、L1(M,N,K)、L0(if different from default),Swizzle、Engineering custom per-step (μs)、Benchmark (μs)、Relative to default Δ% or conclusion(best / fallback / to be retested). Each additional probe append one row, do not overwrite history.
Isolated probing (whole-file replacement of the kernel)	To run only one tile, you may temporarilychange `op_kernel` to a minimal kernel that instantiates all problems;it must be restored before the end of the same round of taskswith shape dispatchproduction version, and recompile and install;forbiddento treat the minimal kernel used for probing as the final deliverable.
Relationship with §6	Before formally adopting a configuration, you should still run the complete JSONL or the user-specified case set PRE/POST(§6.5) and save the report to disk; the single-shape table only records the search process, and does not replace full regression.