external-gitcode-ascend-ascendc-operator-testcase-gen

AscendC Operator Testcase Gen Skill

Generate verification cases and benchmarks based on the operator design document (design.md), for use by subsequent UT code generation, precision verification, and performance verification skills.

Usage Scenarios

In ascendc-operator-design createdesign.mdCall after, before invokingascendc-operator-ut-gen/ascendc-operator-precision-eval/ascendc-operator-performance-eval.

Invocation Interface

Required Parameters

When invoking this skill, the following parameters must be explicitly provided:

Parameter 1: Design Document

Based on the user-provideddesign.mddesign cases
Must include operator logic, inputs and outputs, constraints, and supported data types
If not specified, default to design.md in the project path

Design Process

1. Operator Design Document Parsing

Read the [Operator Interface], [Computation Logic], and [Reference Implementation] sections in the design document
Based on the operator's actual computation logic as well as existing pyTorch implementations, reference documents, etc., generate the operator benchmark and understand the mutual shape constraints among the operator's inputs and outputs.
Based on the constraints, generate typical cases for common shapes when the operator is used in model scenarios, as well as generalized cases generated according to boundary conditions.

2. Test Case Generation

MANDATORY: Before generating test cases, the following reference documents must be read:

Required reading:

templates/test-cases-template.md — Test case format reference
design.md — User-provided operator design document

2.1 Design Principles

Full dtype coverage: Each shape / each boundary value traverses all dtypes supported by the operator,SUPPORTED_DTYPESmust includedesign.mdAll data types supported in [Parameter Description]
Shape determined by the operator: Select appropriate shapes according to the dimensions supported by the operator; do not write fixed dimensions
Shapes should not be too large: Keep the number of elements in a single case within a reasonable range to avoid unnecessary large tensors
Total number of cases = (len(TEST_SHAPES) + len(GENERAL_SHAPES)) × len(SUPPORTED_DTYPES) ≥ 30

Part A: Regular Shape Testing (TEST_SHAPES)

According to the dimensions supported by the operator and the operator type, select suitable shapes from the following dimension pool to form the TEST_SHAPES list.

MUST Select according to the dimensions actually supported by the operator; do not select unsupported dimensions.

Shape Selection Reference Pool

Dimension	Recommended Shape	Applicable Operator Type
1D	(128,), (1024,), (4096,), (8192,)	elementwise, reduction
2D	(32, 512), (64, 768), (128, 1024)	elementwise, matmul, linear
3D	(8, 16, 64), (4, 128, 256)	elementwise, attention, conv1d
4D	(4, 8, 32, 16), (2, 64, 32, 32)	conv2d, elementwise
5D	(2, 3, 4, 5, 6)	conv3d, elementwise

Shapes should not be too large: It is recommended that the number of elements in a single shape be ≤ 200K.

TEST_SHAPES Format

TEST_SHAPES = [
    ("category_name", "description", (dim0, dim1, ...)),
    # ...
]

Category names are user-defined; it is recommended to name them by dimension or scenario. Example:

# elementwise 算子（支持任意维度）
TEST_SHAPES = [
    ("1D",         "128 elements",         (128,)),
    ("1D",         "1024 elements",        (1024,)),
    ("1D",         "4096 elements",        (4096,)),
    ("1D",         "8192 elements",        (8192,)),
    ("2D",         "32x512",              (32, 512)),
    ("3D",         "8x16x64",             (8, 16, 64)),
    ("3D",         "4x128x256",           (4, 128, 256)),
    ("4D",         "4x8x32x16",           (4, 8, 32, 16)),
]

Part B: Generalized Shape Testing (GENERAL_SHAPES)

According to the dimensions supported by the operator and the operator type, select suitable shapes from the following dimension pool to form the GENERAL_SHAPES list.

MUST Select according to the dimensions actually supported by the operator; do not select unsupported dimensions.

Shape Selection Reference Pool

Small Shape Scenarios(boundary testing, extremely small values, non-aligned testing)

Dimension	Small Shape	Testing Purpose
1D	(1,), (2,), (4,), (8,)	Minimal elements, boundary values
1D	(3,), (5,), (7,)	Unaligned length
2D	(1, 1), (2, 2), (4, 4)	Minimal 2D tensor
2D	(1, 128), (2, 256)	Single-row/single-column scenario
3D	(1, 1, 1), (2, 2, 2)	Minimal 3D tensor
3D	(1, 8, 64), (2, 16, 128)	Single-batch scenario

Large Shape scenario(production environment, stress testing, typical model scenarios)

Dimension	Large shape	Applicable model	Number of elements
1D	(3072,), (4096,)	BERT FFN intermediate layer	3K-4K
1D	(5120,), (6400,)	GPT-2 FFN intermediate layer	5K-6K
2D	(512, 768)	BERT-base full sequence	393K
2D	(512, 1024)	BERT-large full sequence	524K
2D	(1024, 768)	GPT-2 sequence	786K
2D	(1024, 1024)	GPT-2 medium sequence	1M
2D	(1024, 1600)	GPT-2 XL sequence	1.6M
3D	(8, 512, 768)	BERT-base batch	3.1M
3D	(8, 197, 768)	ViT-base batch	1.2M
3D	(16, 1024, 1024)	GPT-2 large batch	16.7M

Notes：

The recommended shape for production environments and typical model scenarios should vary according to the specific application scenario of each operator

Small shapes are used for boundary testing to ensure the operator works correctly with minimal inputs

Large shapes are used for performance testing and to verify the correctness of large-scale data

Shapes with more than 200K elements are only used for generalization testing and are not used in regular tests

GENERAL_SHAPES format

GENERAL_SHAPES = [
    ("Small", "description", (dim0, dim1, ...)),
    ("Large", "description", (dim0, dim1, ...)),
    # ...
]

Category is recommended to use "Small" and "Large" to distinguish scenarios. Example:

# elementwise 算子（支持任意维度）
GENERAL_SHAPES = [
    # 小Shape场景
    ("Small", "single element",         (1,)),
    ("Small", "tiny vector 2",          (2,)),
    ("Small", "tiny vector 4",          (4,)),
    ("Small", "unaligned length 3",     (3,)),
    ("Small", "unaligned length 5",     (5,)),
    ("Small", "2x2 matrix",             (2, 2)),
    ("Small", "1x128 single row",       (1, 128)),
    ("Small", "1x1x1 scalar",           (1, 1, 1)),
    ("Small", "1x8x64 single batch",    (1, 8, 64)),
    
    # 大Shape场景（生产环境）
    ("Large", "BERT-base FFN 3072",     (3072,)),
    ("Large", "BERT-large FFN 4096",    (4096,)),
    ("Large", "BERT-base 512x768",      (512, 768)),
    ("Large", "BERT-large 512x1024",    (512, 1024)),
    ("Large", "GPT-2 1024x768",         (1024, 768)),
    ("Large", "GPT-2 1024x1024",        (1024, 1024)),
    ("Large", "ViT-base 8x197x768",     (8, 197, 768)),
]

2.2 Design case output

Based on the collected information, read the templates/test-cases-template.md template, fill in all sections, and output to csrc/ops/[op-name]/test/[op-name]-test-cases.md。

Output location: ascend-kernel/csrc/ops/[op-name]/test/[op-name]-test-cases.md

Notes

Strictly follow the template sections: only output the sections defined in the template (operator benchmark, typical cases, generalization cases, usage instructions)
Do not output irrelevant content：
- ❌ Accuracy verification standards (handled by ascendc-operator-precision-eval )
- ❌ Performance verification standards (handled by ascendc-operator-performance-eval )
- ❌ UT test code (handled by other skills)
- ❌ Any content beyond the template section definitions
Output file location：ascend-kernel/csrc/ops/[op-name]/test/[op-name]-test-cases.md

About this skill

Usage Scenarios

Invocation Interface

Required Parameters

Design Process

1. Operator Design Document Parsing

2. Test Case Generation

2.1 Design Principles

Part A: Regular Shape Testing (TEST_SHAPES)

Shape Selection Reference Pool

TEST_SHAPES Format

Part B: Generalized Shape Testing (GENERAL_SHAPES)

Shape Selection Reference Pool

GENERAL_SHAPES format

2.2 Design case output

Notes