SuperAPI AI Directory
Partner with us中文AI newsFavorites
Skillsexternal-gitcode-ascend-ascendc-operator-performance-eval

external-gitcode-ascend-ascendc-operator-performance-eval

Testing & quality

View repository

Updated 10-15 17:20

About this skill

AscendC operator torch_npu.profiler performance evaluation

Reference files in this skill directory

When executing this skill, you should preferentially use the materials under this directory :

File / Directory Purpose
SKILL.md(this file) Workflow, directory conventions,complete JSONL case specification, report structure,fixed schedule
references/REFERENCE_JSON_CASE_FORMAT.md and the "Performance Case JSONL Specification" belowsame document
references/REFERENCE_PROFILER_AND_METRICS.md torch_npu.profiler、op_statistic.csv、*_ascend_pt path
examples/sample_perf_cases.jsonl Minimal LayerNorm-style JSONL, can be copied and renamed
examples/layer_norm_profiler_reference/ Layer Norm reference implementation（layer_norm_profiler_common.py、benchmark_layer_norm_torch_npu_profiler.py, case JSONL, documentation); for a new operator, you can copy this directory to csrc/ops/<op>/test/ and then replace the forward pass and file names

Role

In ascend-kernel , establish a reusable csrc/ops/<算子名>/ for profiler performance case and a custom operator vs baseline Markdown report workflow. Collection must go through torch_npu.profiler, and warmup and active are fixed at 5(see the next section). For details, see references/REFERENCE_PROFILER_AND_METRICS.md。

Core principles (two mandatory constraints)：

  1. The comparison report must always present: regardless of whether the baseline path is a baseline API or a small-operator composition, the final report must include a dual-path comparison table of custom operator vs baseline,single-path reports are prohibited as a substitute，The baseline must run on the NPU。
  2. Case generation must first read design.md: before generating any JSONL case, you must read the csrc/ops/<op>/design.mdunder the operator directory, and extract from it the parameter constraints, typical shapes, supported dtypes, and key attribute values; the cases must cover all execution modes described in the design document.

Case source: load from the testcase-gen case document (MANDATORY)

Before generating or modifying any JSONL case，MUST , first read the case document produced by testcase-gen:

Step 0: Read the testcase-gen case document

READ csrc/ops/<op>/test/<op>-test-cases.md

Extract from it:

Extraction item Location in the case document Purpose
SUPPORTED_DTYPES §Test configuration Dtype coverage of the JSONL cases
TEST_SHAPES §Test configuration Basis for selecting small/medium/large-scale shapes
GENERAL_SHAPES §Test configuration Generalized shapes, which can be added for performance scenarios
NPU invocation method §Operator baseline Forward invocation of the custom operator
CPU reference implementation §Operator baseline Reference implementation of the baseline path

testcase-gen output → JSONL case conversion rules

  1. Select from TEST_SHAPES + GENERAL_SHAPESrepresentative shapes(covering small/medium/large scale), avoiding duplication
  2. For each shape, iterate over all dtypes in SUPPORTED_DTYPES
  3. Populate the JSONL's inputs field with attribute values from design.md (such as block_size, eps, etc.)
  4. Total number of JSONL test cases ≥ 8
  5. The NPU invocation method and CPU reference implementation in the operator benchmark are used to construct the custom operator path and the benchmark path

If <op>-test-cases.md does not exist: fall back to designing test cases entirely from design.md on your own (following the process below), but the report must note "test cases are self-designed, not produced by testcase-gen".

Test case generation: design.md must be read first (mandatory)

Before generating or modifying any JSONL test case(regardless of whether the testcase-gen test case document has been loaded), the following steps must be performed:

Step 1: Read the design document

READ csrc/ops/<op>/design.md

Extract from it:

Extraction item Location in design.md Purpose
Supported data types §1 "Supported data types" dtype coverage of the test cases
Parameter constraints and value ranges §1 "Parameter description" constraints column Legal range of attribute values (such as block_size ≤ 128)
Typical shape / input size §2 "Computation logic" / §3 "Tiling strategy" Shape baseline for small/medium/large-scale test cases
Mode combinations of key attributes §2 "Pseudocode" / §1 "Parameter description" Execution paths that each need to cover (such as do_transpose=True/False, is_input_split=True/False)
Performance-critical points §6 "Performance optimization" / §3 "Tiling strategy" Branches that affect performance (such as transpose vs non-transpose taking different DMA paths)

Step 2: Test case design rules

Rule Description
Cover all execution modes When design.md describes multiple execution paths (such as transpose/non-transpose, input_split modes), each mode must have at least one test case
Cover all supported dtypes Each supported data type must have at least one set of test cases, typically with a medium-scale shape
One set each for small/medium/large-scale shapes Small scale (kernel launch overhead dominant), medium scale (typical production scenario), and large scale (memory access bandwidth dominant) must each be covered
Parameter values come from the constraint range Attribute values (such as block_size) must be selected from the constraints in design.md and must not be set arbitrarily
Integer/index tensor values must be semantically legal The specific values of tensors such as win_lengths and offsets must satisfy operator semantics (such as offsets must be legal window start offsets)

Step 3: Validate the test cases

After generating test cases, check:

  • All dtypes are covered
  • All execution modes (defined by design.md) have corresponding test cases
  • Parameter values (including attribute values and integer tensor values) are within the constraints of design.md
  • Contains at least one "small shape" test case and one "large shape" test case

Reference path decision tree (mandatory)

Performance evaluationAlwaysA dual-path comparison is required (custom operator vs. benchmark), and the benchmark path is determined in the following order:

算子是否有标杆等价 API？
  ├─ 是（如 torch.nn.functional.*、torch_npu 内置算子）
  │    └─ 使用标杆 API 作为标杆路径
  └─ 否（无标杆等价接口）
       └─ 必须实现「小算子拼接」标杆路径 ← 本技能的强制要求
            └─ 用设计文档 §「参考实现」或「伪代码」中的 PyTorch 基础算子组合实现

Requirements for the benchmark path of small-operator stitching

When there is no benchmark-equivalent interface,you must：

  1. Read the reference implementation from design.md: the design document usually contains a PyTorch reference implementation (pseudocode or a Python function); build the benchmark path based on that implementation.
  2. Use combinations of PyTorch basic operators：torch.zeros, slice assignment,.permute()、torch.cat , etc. Do not use loops + Python scalar assignment (otherwise the profiler collects CPU operators rather than NPU operators, making a fair comparison impossible); the entire benchmark implementation must be based mainly on tensor operations and be executable on the NPU.
  3. Clearly mark in the report: the report header must state "no benchmark-equivalent interface; the benchmark path is small-operator stitching" and list the basic operators used.
  4. The comparison table must present: do not degrade to a single-path report on the grounds of "no benchmark interface"; you must retain the three columns "custom operator per-step", "benchmark per-step", and "ratio".

NEVER: outputting a single-path report or skipping the comparison table on the grounds of "no benchmark-equivalent interface".

Fixed Profiler step count (mandatory)

Parameter Value Description
warmup 5 Scripts or the CLI are not allowed to change it to other values
active 5 Scripts or the CLI are not allowed to change it to other values
wait Default 0 The CLI or a constant may be retained, as needed
repeat Default 1 For simple scenarios, fix it to 1; if repeat>1, the CSV selection semantics must be explained in the documentation

At the end of each step, you must prof.step(); total loop steps = repeat * (wait + warmup + active)。

File locations (uniformly under the operator test/ subdirectory)

All of the following artifacts are placed in ascend-kernel/csrc/ops/<op>/test/：

Category Naming convention (<op> such as layer_norm）
Test case JSONL only <op>_perf_cases.jsonl(one JSON object per line);not maintained, not generated <op>_perf_cases.json
Markdown report <op>_torch_npu_profiler_report.md(the only structured result persisted;not generated <op>_torch_npu_profiler_results.json）
Profiler export root directory test/profiler_trace/(or --trace-root override)

The performance script, common modules, and the above files are located together test/。

Full specification for the performance test case JSONL

The following is the complete field and type description for the test case file;use only .jsonl as the test case carrier.

1. File form

Form Description
JSONL Each line one JSON object, with a newline at the end of the line; blank lines are ignored. Extension .jsonl。

ProhibitedIn this process, generate or maintain in sync with the test cases .json array files.

2. Top-level structure of a single test case

Each test case object must contain the key "inputs", with value array。

Layer Norm example(if the operator differs, replace build_inputs convention; the structure must still contain inputs the array):

{
  "inputs": [
    { "name": "x", "type": "tensor", "required": true, "dtype": "float16", "shape": [8, 128] },
    { "name": "normalized_shape", "type": "attr", "required": true, "dtype": "int", "value": [128] },
    { "name": "use_affine", "type": "attr", "required": false, "dtype": "bool", "value": true },
    { "name": "eps", "type": "attr", "required": false, "dtype": "float", "value": 1e-05 }
  ]
}
  • inputs Each element within name must be unique within the same use case。
  • Other operators: tensor / tensor_list / attr / integer tensor range and other rules see references/REFERENCE_JSON_CASE_FORMAT.md。

3. Complete JSONL example (two lines, Layer Norm)

{"inputs":[{"name":"x","type":"tensor","required":true,"dtype":"float16","shape":[2,128]},{"name":"normalized_shape","type":"attr","required":true,"dtype":"int","value":[128]},{"name":"use_affine","type":"attr","required":false,"dtype":"bool","value":true},{"name":"eps","type":"attr","required":false,"dtype":"float","value":1e-05}]}
{"inputs":[{"name":"x","type":"tensor","required":true,"dtype":"float16","shape":[4,256]},{"name":"normalized_shape","type":"attr","required":true,"dtype":"int","value":[256]},{"name":"use_affine","type":"attr","required":false,"dtype":"bool","value":false},{"name":"eps","type":"attr","required":false,"dtype":"float","value":1e-05}]}

More complete field descriptions (tensor_list、int tensor range , etc.) see references/REFERENCE_JSON_CASE_FORMAT.md。

Profiler and directory semantics (summary)

  • Each time with torch_npu.profiler.profile(...), under the handler directory generate with _ascend_pt as the suffix export directory; CSV path …/*_ascend_pt/ASCEND_PROFILER_OUTPUT/op_statistic.csv。
  • Each use case, each implementation(e.g. custom / baseline）one independent with; subpath recommended {trace_root}/{op_trace_tag}/{custom|baseline}/case_XXX/, clear before running case_XXX。

See references/REFERENCE_PROFILER_AND_METRICS.md。

Metrics (summary)

  1. For a single with corresponding CSV:Sum Total Time(us) of each operator row。
  2. divided by active×repeat（divisor_mode=active_steps when) or only active（active_only). This skill fixes active=5; if repeat=1, then divisor = 5。

Required structure of the performance comparison report (Markdown)

The report format strictly follows examples/sample_report.md, with the following structure:

1. Title

# 性能评估结果

2. Comparison table (unified single table, mandatory dual paths)

All use cases are displayed inthe same table, with the header fixed as Case | Shape | DType | 自定义算子(us) | 标杆(us) | 加速比。

Example:

## 性能对比

| Case | Shape | DType | 自定义算子(us) | 标杆(us) | 加速比 |
| ---- | ----- | ----- | ------------- | -------- | -------------- |
| 0 | [128, 4096] | float16 | 9.75 | 10.10 | 1.036 |
| 1 | [128, 5120] | float16 | 10.52 | 9.39 | 0.893 |
| 2 | [128, 6144] | float16 | 10.99 | 14.36 | 1.307 |
| 3 | [64, 6400] | float16 | 9.13 | 9.49 | 1.040 |
| 4 | [2, 1024, 4096] | float16 | 57.01 | 84.92 | 1.490 |
| 5 | [2, 1024, 6144] | float16 | 73.80 | 139.56 | 1.891 |
| 6 | [1, 2048, 6400] | float16 | 75.60 | 143.09 | 1.893 |
| 7 | [64, 4096] | float32 | 8.45 | 7.14 | 0.846 |

3. Full summary

Use ## 全量汇总 second-level heading, containing akey-value pair table：

## 全量汇总

| 指标 | 值 |
| ---- | -- |
| 用例数 | N |
| 平均 加速比（>1 表示自定义算子更快） | X.XXX |
| 自定义算子更优（比值>1） | M |
| 标杆更优（比值<1） | K |

Immediately below, use ### 按数据类型汇总 third-level heading to display the summary table by dtype:

### 按数据类型汇总

| DType | 用例数 | 平均 加速比 | 自定义算子更优 | 标杆更优 |
| ----- | ------ | ------------------- | ------------- | -------- |
| float16 | 7 | 1.364 | 6 | 1 |
| float32 | 1 | 0.846 | 0 | 1 |

4. Brief analysis

Use ## 简短分析 second-level heading, list ≥3 brief conclusions in the form of an unordered list, covering: overall trends, differences across dtype / shape scales, memory access and compute characteristics, etc.

## 简短分析

- 平均 加速比 大于 1，自定义算子整体略有优势。
- 大 shape 下自定义算子优势更明显，向量路径利用更充分。
- float32 小 shape 场景自定义算子略逊于标杆，可能与 kernel launch 开销占比较高有关。

Other conventions

  • Do not writethe *_profiler_results.jsonthat duplicates the report; intermediate statistics exist only in script memory and are written to Markdown.

Display results in the conversation (MANDATORY)

Generate csrc/ops/<op>/test/<op>_torch_npu_profiler_report.md(or it already exists and has been updated by this run), then in the reply ofthe current conversationthe assistant MUST simultaneously completes the following, andmust notoutput only "report generated" and the path without displaying the data:

  1. Paste the main performance content(the user can read the conclusions without opening the file; the displayed content is consistent with the report structure):
    • Unified comparison table: header Case | Shape | DType | 自定义算子(us) | 标杆(us) | 加速比, all dtypes displayed in the same table. When there are many cases, it may be truncated with a note "see the report for the rest".
    • Full summary: summary metrics in key-value pair form (number of use cases, average ratio, number of cases where the custom operator/baseline is better) and the summary table by data type.
    • Brief analysis：≥3 conclusions in an unordered list, covering overall trends, differences across dtype / shape scales, memory access and compute characteristics, etc.

NEVER: reply only with the report path; NEVER replace displaying the core numbers and conclusions in the conversation with "please open the Markdown yourself".

Common pitfalls

  • warmup / active was changed to non-5, inconsistent with the skill convention.
  • Did not use torch_npu.profiler or prof.step() , inconsistent with schedule.
  • repeat>1 may have multiple *_ascend_pt exports; when taking CSV by mtime, the semantics must be clarified.
  • When CSV header BOM / column names change, must be compatible with Total Time(us)。
  • When a custom operator is not registered, only perform the baseline path; the custom library must be loaded before comparison.
  • Did not read <op>-test-cases.md and directly designed test cases on its own: testcase-gen has already generated a unified test case document, so shape and dtype should preferably be extracted from it to avoid duplicate design.
  • Generated test cases directly without reading design.md: resulting in shapes that do not meet constraints and missing coverage of key execution modes (e.g., missing the transpose=True path).
  • Output a single-path report on the grounds of "no baseline equivalent interface": a small-operator concatenated baseline path must be implemented, and a dual-path comparison table must always be output.
  • Small-operator concatenation uses Python loops for scalar-by-scalar assignment: what the profiler collects is CPU logic rather than NPU operators, causing the baseline path timing to be distorted; the baseline implementation should be primarily tensor operations.
  • The assistant only outputs the report path and does not display the main tables and summary conclusions in the current conversation(violates "display results within the conversation").

Reference implementation (examples/layer_norm_profiler_reference/）

and ascend-kernel/csrc/ops/layer_norm/test/ profiler-related files remain isomorphic, including:

  • layer_norm_profiler_common.py、benchmark_layer_norm_torch_npu_profiler.py
  • layer_norm_perf_cases.jsonl(JSONL only, no .json）
  • LAYER_NORM_PROFILER_PERF_GUIDE.md、README.md

For a new operator, copy the entire directory from copy the entire directory to csrc/ops/<op>/test/, then replace the operator name, forward call,build_inputs and trace subdirectory name. If layer_norm/test/ in the repository has updates, they should be synced back to examples/layer_norm_profiler_reference/。

Checklist (assistant self-check)

  • has read csrc/ops/<op>/test/<op>-test-cases.md(if it exists), and extracted SUPPORTED_DTYPES, TEST_SHAPES, GENERAL_SHAPES, operator baseline from it
  • has read csrc/ops/<op>/design.md, and extracted dtype, parameter constraints, typical shapes, execution modes from it
  • Test cases coverall execution modesdescribed in design.md (e.g., transpose/non-transpose, input_split mode, etc.)
  • Parameter values in test cases (attribute values and integer tensor values) are all within the constraints of design.md
  • The baseline path type (baseline API or small-operator concatenation) has been confirmed and clearly marked at the top of the report
  • If there is no baseline equivalent interface, has implementedsmall-operator concatenationbaseline path, and the baseline implementation is primarily tensor operations (not Python scalar loops)
  • has used torch_npu.profiler, and warmup=5、active=5 has not been rewritten
  • has generated or updated <op>_torch_npu_profiler_report.md, with a format consistent with examples/sample_report.md : a unified comparison table including a DType column + full summary + summary by data type + brief analysis
  • has been displayed in the current conversationa unified comparison table including a DType column, full summary, summary by data type, and ≥3 itemsbrief analysis conclusions, not just the path
  • has explained the fixed-step convention and metric definitions