SuperAPI AI Directory
Partner with us中文AI newsFavorites
Skillsascendc-operator-performance-eval

ascendc-operator-performance-eval

Testing & quality

View repository

Updated 10-16 05:00

About this skill

AscendC operator torch_npu.profiler performance evaluation

Reference files in this skill directory

When executing this skill, priority should be given to using materials under this directory :

File / Directory Purpose
SKILL.md(this file) Process, directory conventions,complete JSONL test case specification, report structure,fixed schedule
references/REFERENCE_JSON_CASE_FORMAT.md and "Performance Test Case JSONL Specification" belowsame text
references/REFERENCE_PROFILER_AND_METRICS.md torch_npu.profiler、op_statistic.csv、*_ascend_pt path
examples/sample_perf_cases.jsonl Minimal LayerNorm-style JSONL, can be copied and renamed
examples/layer_norm_profiler_reference/ Layer Norm reference implementation（layer_norm_profiler_common.py、benchmark_layer_norm_torch_npu_profiler.py, test case JSONL, description); for a new operator, copy this directory to csrc/ops/<op>/test/ then replace the forward and file names

Role

In ascend-kernel , for csrc/ops/<算子名>/ establish a reusable profiler performance test case and custom operator vs baseline Markdown report process. Collection must go through torch_npu.profiler, and warmup and active are fixed at 5(see next section). For details see references/REFERENCE_PROFILER_AND_METRICS.md。

Core principles (two mandatory constraints)：

  1. Comparison report must always be presented: regardless of whether the baseline path is a baseline API or small-operator composition, the final report must include a custom operator vs baseline dual-path comparison table,single-path reports are prohibited as substitutes，The baseline must run on NPU。
  2. Test case generation must first read design.md: before generating any JSONL test case, you must read the csrc/ops/<op>/design.mdin the operator directory, extract parameter constraints, typical shapes, supported dtypes and key attribute values from it; test cases must cover all execution modes described in the design document.

Test case source: load from testcase-gen test case document (MANDATORY)

Before generating or modifying any JSONL test case，MUST first read the test case document produced by testcase-gen:

Step 0: Read the testcase-gen test case document

READ csrc/ops/<op>/test/<op>-test-cases.md

Extract from it:

Extraction item Location in the test case document Purpose
SUPPORTED_DTYPES §Test configuration dtype coverage of JSONL test cases
TEST_SHAPES §Test configuration Basis for selecting small/medium/large-scale shapes
GENERAL_SHAPES §Test configuration Generalized shapes, may be supplemented for performance scenarios
NPU invocation method §Operator baseline Forward invocation of the custom operator
CPU reference implementation §Operator baseline Reference implementation of the baseline path

testcase-gen output → JSONL test case conversion rules

  1. Select from TEST_SHAPES + GENERAL_SHAPESrepresentative shapes(covering small/medium/large scale), avoiding duplicates
  2. For each shape, iterate over all dtypes in SUPPORTED_DTYPES
  3. Populate the JSONL's inputs field using the attribute values in design.md (such as block_size, eps, etc.)
  4. Total number of JSONL cases ≥ 8
  5. The NPU invocation method and CPU reference implementation in the operator benchmark are used to build the custom operator path and the benchmark path

If <op>-test-cases.md does not exist: fall back to designing cases entirely from design.md on your own (following the process below), but you must note in the report that "the cases are self-designed and not produced by testcase-gen".

Case generation: design.md must be read first (mandatory)

Before generating or modifying any JSONL case(regardless of whether the testcase-gen case document has been loaded), the following steps must be performed:

Step 1: Read the design document

READ csrc/ops/<op>/design.md

Extract from it:

Extraction item Location in design.md Purpose
Supported data types §1 "Supported data types" dtype coverage of the cases
Parameter constraints and value ranges §1 "Parameter description" constraints column Valid range of attribute values (such as block_size ≤ 128)
Typical shape / input scale §2 "Computation logic" / §3 "Tiling strategy" Shape baseline for small/medium/large-scale cases
Mode combinations of key attributes §2 "Pseudocode" / §1 "Parameter description" Execution paths that need to be covered separately (such as do_transpose=True/False, is_input_split=True/False)
Performance-critical points §6 "Performance optimization" / §3 "Tiling strategy" Branches that affect performance (such as transpose vs non-transpose taking different DMA paths)

Step 2: Case design rules

Rule Description
Cover all execution modes When design.md describes multiple execution paths (such as transpose/non-transpose, input_split mode), each mode must have at least one case
Cover all supported dtypes Each supported data type must have at least one set of cases, typically a medium-scale shape
One set each for small/medium/large-scale shapes Small-scale (dominated by kernel launch overhead), medium-scale (typical production scenario), and large-scale (dominated by memory access bandwidth) must each be covered
Parameter values come from the constraint range Attribute values (such as block_size) must be selected from the constraints in design.md and must not be set arbitrarily
Integer/index tensor values must be semantically valid The specific values of tensors such as win_lengths and offsets must satisfy the operator semantics (such as offsets must be valid window start offsets)

Step 3: Validate the cases

After generating the cases, check:

  • All dtypes are covered
  • All execution modes (as defined by design.md) have corresponding cases
  • Parameter values (including attribute values and integer tensor values) are within the constraints of design.md
  • Include at least one "small shape" case and one "large shape" case

Reference path decision tree (mandatory)

Performance evaluationAlwaysA dual-path comparison is required (custom operator vs. benchmark), and the benchmark path is determined in the following order:

算子是否有标杆等价 API？
  ├─ 是（如 torch.nn.functional.*、torch_npu 内置算子）
  │    └─ 使用标杆 API 作为标杆路径
  └─ 否（无标杆等价接口）
       └─ 必须实现「小算子拼接」标杆路径 ← 本技能的强制要求
            └─ 用设计文档 §「参考实现」或「伪代码」中的 PyTorch 基础算子组合实现

Requirements for the benchmark path of small-operator stitching

When there is no benchmark-equivalent interface,you must：

  1. Read the reference implementation from design.md: the design document usually contains a PyTorch reference implementation (pseudocode or Python function); build the benchmark path based on that implementation.
  2. Use combinations of basic PyTorch operators：torch.zeros, slice assignment,.permute()、torch.cat , etc. Do not use loops + Python scalar assignment (otherwise the profiler collects CPU operators rather than NPU operators, making a fair comparison impossible); the entire benchmark implementation must be based primarily on tensor operations and be executable on the NPU.
  3. Clearly state in the report: the report header must state "No benchmark-equivalent interface; the benchmark path is small-operator stitching" and list the basic operators used.
  4. The comparison table must present: do not degrade to a single-path report on the grounds of "no benchmark interface"; you must retain the three columns "custom operator per-step", "benchmark per-step", and "ratio".

NEVER: output a single-path report or skip the comparison table on the grounds of "no benchmark-equivalent interface".

Fixed profiler step count (mandatory)

Parameter Value Description
warmup 5 Scripts or the CLI are not allowed to change it to other values
active 5 Scripts or the CLI are not allowed to change it to other values
wait Default 0 CLI or constant may be retained, as needed
repeat Default 1 For simple scenarios, fix it at 1; if repeat>1, the CSV selection semantics must be explained in the documentation

At the end of each step, you must prof.step(); total number of loop steps = repeat * (wait + warmup + active)。

File placement (uniformly in the operator's test/ subdirectory)

Place all of the following artifacts in ascend-kernel/csrc/ops/<op>/test/：

Category Naming convention (<op> such as layer_norm）
Use case JSONL only <op>_perf_cases.jsonl(one JSON object per line);do not maintain, do not generate <op>_perf_cases.json
Markdown report <op>_torch_npu_profiler_report.md(the only structured result written to disk;do not generate <op>_torch_npu_profiler_results.json）
Profiler export root directory test/profiler_trace/(or --trace-root override)

Performance scripts, common modules, and the above files in the same place test/。

Complete specification of the performance use case JSONL

The following is the complete field and type description for the use case file;use only .jsonl as the use case carrier.

1. File format

Format Description
JSONL Each line one JSON object, with a newline at the end of the line; blank lines are ignored. Extension .jsonl。

ProhibitedIn this process, generate or maintain in sync with the use cases .json array files.

2. Top-level structure of a single use case

Each use case object must contain the key "inputs", whose value is a array。

Layer Norm example(if the operator differs, replace build_inputs convention; the structure must still contain the inputs array):

{
  "inputs": [
    { "name": "x", "type": "tensor", "required": true, "dtype": "float16", "shape": [8, 128] },
    { "name": "normalized_shape", "type": "attr", "required": true, "dtype": "int", "value": [128] },
    { "name": "use_affine", "type": "attr", "required": false, "dtype": "bool", "value": true },
    { "name": "eps", "type": "attr", "required": false, "dtype": "float", "value": 1e-05 }
  ]
}
  • inputs elements within name are unique within the same use case。
  • Other operators: tensor / tensor_list / attr / integer tensor range and other rules see references/REFERENCE_JSON_CASE_FORMAT.md。

3. Complete JSONL example (two lines, Layer Norm)

{"inputs":[{"name":"x","type":"tensor","required":true,"dtype":"float16","shape":[2,128]},{"name":"normalized_shape","type":"attr","required":true,"dtype":"int","value":[128]},{"name":"use_affine","type":"attr","required":false,"dtype":"bool","value":true},{"name":"eps","type":"attr","required":false,"dtype":"float","value":1e-05}]}
{"inputs":[{"name":"x","type":"tensor","required":true,"dtype":"float16","shape":[4,256]},{"name":"normalized_shape","type":"attr","required":true,"dtype":"int","value":[256]},{"name":"use_affine","type":"attr","required":false,"dtype":"bool","value":false},{"name":"eps","type":"attr","required":false,"dtype":"float","value":1e-05}]}

More complete field descriptions (tensor_list、int tensor range etc.) see references/REFERENCE_JSON_CASE_FORMAT.md。

Profiler and directory semantics (summary)

  • Each time with torch_npu.profiler.profile(...), under the handler directory generate with _ascend_pt as the suffix export directory; CSV path …/*_ascend_pt/ASCEND_PROFILER_OUTPUT/op_statistic.csv。
  • Each use case, each implementation(e.g. custom / baseline）one independent with; sub-path recommended {trace_root}/{op_trace_tag}/{custom|baseline}/case_XXX/, clear before running case_XXX。

See references/REFERENCE_PROFILER_AND_METRICS.md。

Metrics (summary)

  1. For a single with corresponding CSV:sum of Total Time(us) for each operator row。
  2. divided by active×repeat（divisor_mode=active_steps when) or only active（active_only). This skill fixes active=5; if repeat=1, then divisor = 5。

Performance comparison report (Markdown) required structure

The report format strictly follows examples/sample_report.md, the structure is as follows:

1. Title

# 性能评估结果

2. Comparison table (unified single table, mandatory dual path)

All use cases are shown inthe same table, with the header fixed as Case | Shape | DType | 自定义算子(us) | 标杆(us) | 加速比。

Example:

## 性能对比

| Case | Shape | DType | 自定义算子(us) | 标杆(us) | 加速比 |
| ---- | ----- | ----- | ------------- | -------- | -------------- |
| 0 | [128, 4096] | float16 | 9.75 | 10.10 | 1.036 |
| 1 | [128, 5120] | float16 | 10.52 | 9.39 | 0.893 |
| 2 | [128, 6144] | float16 | 10.99 | 14.36 | 1.307 |
| 3 | [64, 6400] | float16 | 9.13 | 9.49 | 1.040 |
| 4 | [2, 1024, 4096] | float16 | 57.01 | 84.92 | 1.490 |
| 5 | [2, 1024, 6144] | float16 | 73.80 | 139.56 | 1.891 |
| 6 | [1, 2048, 6400] | float16 | 75.60 | 143.09 | 1.893 |
| 7 | [64, 4096] | float32 | 8.45 | 7.14 | 0.846 |

3. Full summary

Use ## 全量汇总 second-level heading, containing akey-value pair table：

## 全量汇总

| 指标 | 值 |
| ---- | -- |
| 用例数 | N |
| 平均 加速比（>1 表示自定义算子更快） | X.XXX |
| 自定义算子更优（比值>1） | M |
| 标杆更优（比值<1） | K |

Immediately below, use ### 按数据类型汇总 third-level heading, showing the summary table by dtype:

### 按数据类型汇总

| DType | 用例数 | 平均 加速比 | 自定义算子更优 | 标杆更优 |
| ----- | ------ | ------------------- | ------------- | -------- |
| float16 | 7 | 1.364 | 6 | 1 |
| float32 | 1 | 0.846 | 0 | 1 |

4. Brief analysis

Use ## 简短分析 second-level heading, list ≥3 brief conclusions in unordered list form, covering: overall trends, differences across dtype / shape scales, memory access and compute characteristics, etc.

## 简短分析

- 平均 加速比 大于 1，自定义算子整体略有优势。
- 大 shape 下自定义算子优势更明显，向量路径利用更充分。
- float32 小 shape 场景自定义算子略逊于标杆，可能与 kernel launch 开销占比较高有关。

Other conventions

  • Do not writecontent that duplicates the report *_profiler_results.json; intermediate statistics exist only in script memory and are written to Markdown.

Show results in the conversation (MANDATORY)

After generating csrc/ops/<op>/test/<op>_torch_npu_profiler_report.md(or if it already exists and has been updated in this run), the assistant inthe current conversation's reply MUST simultaneously completes the following,must notoutput only "report generated" and the path without showing the data:

  1. Paste the main performance content(the user can read the conclusions without opening the file, and the displayed content is consistent with the report structure):
    • Unified comparison table: header Case | Shape | DType | 自定义算子(us) | 标杆(us) | 加速比, all dtypes are shown in the same table. When there are many cases, it may be truncated with a note "see report for the rest".
    • Full summary: summary metrics in key-value pair form (number of use cases, average ratio, number of cases where the custom operator/baseline is better) and a summary table by data type.
    • Brief analysis：≥3 unordered list conclusions, covering overall trends, differences across dtype / shape scales, memory access and compute characteristics, etc.

NEVER: reply only with the report path; NEVER replace showing the core numbers and conclusions in the conversation with "please open the Markdown yourself".

Common pitfalls

  • warmup / active was changed to a value other than 5, which is inconsistent with the skill convention.
  • Did not use torch_npu.profiler or prof.step() , which is inconsistent with schedule.
  • repeat>1 Possibly multiple *_ascend_pt exports; when selecting CSV by mtime, the semantics need to be explained.
  • When the CSV header BOM / column names change, it must be compatible with Total Time(us)。
  • When a custom operator is not registered, only follow the benchmark path; the custom library must be loaded before comparison.
  • Did not read <op>-test-cases.md and directly designed test cases on your own: testcase-gen has already generated a unified test case document, and shapes and dtypes should be extracted from it first to avoid duplicate design.
  • Generated test cases directly without reading design.md: This caused shapes to not meet the constraints and missing coverage of key execution modes (such as missing the transpose=True path).
  • Output a single-path report on the grounds of "no equivalent benchmark interface": A small-operator concatenated benchmark path must be implemented, and a dual-path comparison table must always be output.
  • Small-operator concatenation used a Python loop with scalar-by-scalar assignment: What the profiler collects is CPU logic rather than NPU operators, causing the benchmark path timing to be distorted; the benchmark implementation should be based mainly on tensor operations.
  • The assistant only output the report path and did not display the main tables and summary conclusions in the current conversation(violating "display results within the conversation").

Reference implementation (examples/layer_norm_profiler_reference/）

and ascend-kernel/csrc/ops/layer_norm/test/ The profiler-related files remain isomorphic, including:

  • layer_norm_profiler_common.py、benchmark_layer_norm_torch_npu_profiler.py
  • layer_norm_perf_cases.jsonl(JSONL only, no .json）
  • LAYER_NORM_PROFILER_PERF_GUIDE.md、README.md

For a new operator, from this directory copy the whole thing to csrc/ops/<op>/test/, then replace the operator name, forward call,build_inputs and trace subdirectory name. If the repository's layer_norm/test/ has updates, they should be synchronized back to examples/layer_norm_profiler_reference/。

Checklist (assistant self-check)

  • Has read csrc/ops/<op>/test/<op>-test-cases.md(if it exists), and extracted SUPPORTED_DTYPES, TEST_SHAPES, GENERAL_SHAPES, and operator benchmarks from it
  • Has read csrc/ops/<op>/design.md, and extracted dtype, parameter constraints, typical shapes, and execution modes from it
  • Test cases cover the design.md-describedall execution modes(such as transpose/non-transpose, input_split mode, etc.)
  • The parameter values (attribute values and integer tensor values) in the test cases are all within the constraints of design.md
  • Has confirmed the benchmark path type (benchmark API or small-operator concatenation), and clearly marked it at the top of the report
  • If there is no equivalent benchmark interface, has implemented asmall-operator concatenatedbenchmark path, and the benchmark implementation is based mainly on tensor operations (not a Python scalar loop)
  • Has used torch_npu.profiler, and warmup=5、active=5 has not been rewritten
  • Has generated or updated <op>_torch_npu_profiler_report.md, with a format consistent with examples/sample_report.md : a unified comparison table including a DType column + full summary + summary by data type + brief analysis
  • has been displayed in the current conversationa unified comparison table including a DType column, full summary, summary by data type, and ≥3 itemsbrief analysis conclusions, not just the path
  • Has explained the fixed-step convention and metric definitions