SuperAPI AI Directory
Partner with us中文AI newsFavorites
Skillsascendc-operator-design

ascendc-operator-design

Design & creation

View repository

Updated 10-15 19:58

About this skill

AscendC Operator Design Skill

Generate a complete design document (design.md) based on the operator requirements, for consumption by the subsequent code-gen skill.

Usage Scenarios

Call this after creating the skeleton in ascendc-operator-project-init and beforeascendc-operator-code-gen generating the code.

Design Process

1. Operator Requirements Analysis

If called by the orchestration skill, the operator name and function description have already been determined; go directly to step 2.

If called standalone, confirm the following information with the user:

Information Required Description
Operator name (snake_case) Yes e.g. acosh, rms_norm
Function description / mathematical formula Yes e.g. "acosh(x) = ln(x + sqrt(x²-1))"
Supported data types No Default float16 + float32

MANDATORY: Check whether PyTorch / NumPy has an interface with the same name. If it exists, the interface signature and semantics Must be aligned with it (e.g. torch.acosh、torch.softmax）。

2. Choose the Implementation Path

Based on the operator characteristics, recommend a suitable implementation approach:

Implementation Path Applicable Scenarios Criteria
AscendC Kernel Pure vector operator Does not involve matrix multiplication
CATLASS template library GEMM / FlashAttention Involves cube matrix computation
ACLNN wrapper CANN already has a built-in operator No custom kernel needed

New operators use the AscendC Kernel path by default, unless matrix multiplication is explicitly involved.

3. Detailed Design Document Generation

MANDATORY: Before generating the design document, the following reference documents must be read:

  1. Required reading: templates/design-template.md — Design document template
  2. Optional reading by operator type:
    • Element-wise operations (add/relu/acosh/sigmoid...) → references/elementwise-tiling.md
    • Reduction operations (softmax/layernorm...) → references/reduction-tiling.md
    • Index operations (gather/index_select/scatter...) → references/index-tiling.md
    • Sort operations (sort/topk...) → references/sort-tiling.md
    • Pooling operations (avgpool/maxpool/...) → references/pooling-tiling.md
  3. General reference: references/general-tiling-principles.md

Never skip reading the reference documents.

3.1 Design Document Structure

The design document contains the following core sections:

  1. Operator Interface Definition — Function signature, parameter descriptions, supported data types
  2. Computation Logic Design — Algorithm description,AscendC API call pseudocode, implementation path selection
  3. Tiling Strategy — Two-level Tiling design (Block level + UB level), UB allocation table, tileLength calculation
  4. Workspace Requirements — Workspace size calculation
  5. Performance Optimization — Key optimization points, operator characteristic analysis
  6. Kernel-side implementation key points — Offset calculation, execution flow,FP16/BF16 precision promotion flow
  7. Implementation checklist — File structure, code key points, testing key points

3.2 Computation logic pseudocode (key output)

Must decompose the mathematical formula into an AscendC API call sequence. This is the direct input for the code-gen skill.

Mapping of common mathematical functions to AscendC APIs：

Mathematical operation AscendC API Remarks
x + y Add(dst, src0, src1, len) Dual input
x - y Sub(dst, src0, src1, len) Dual input
x * y Mul(dst, src0, src1, len) Dual input
x / y Div(dst, src0, src1, len) Dual input
x + scalar Adds(dst, src, scalar, len) Scalar operation, prefer using
x * scalar Muls(dst, src, scalar, len) Scalar operation, prefer using
abs(x) Abs(dst, src, len)
exp(x) Exp(dst, src, len)
ln(x) Ln(dst, src, len)
sqrt(x) Sqrt(dst, src, len)
1/x Reciprocal(dst, src, len)
1/sqrt(x) Rsqrt(dst, src, len)
tanh(x) Tanh(dst, src, len)
relu(x) Relu(dst, src, len)
max(x,y) Max(dst, src0, src1, len)
min(x,y) Min(dst, src0, src1, len)
fp16→fp32 Cast(dst, src, CAST_NONE, len) Precision promotion is lossless
fp32→fp16 Cast(dst, src, CAST_ROUND, len) Precision reduction is lossy

Example — API call sequence for acosh(x)：

Mul(tmp, x, x, len);        // tmp = x²
Adds(tmp, tmp, -1.0f, len); // tmp = x² - 1
Sqrt(tmp, tmp, len);         // tmp = sqrt(x² - 1)
Add(tmp, tmp, x, len);      // tmp = x + sqrt(x² - 1)
Ln(y, tmp, len);             // y = ln(x + sqrt(x² - 1))

Note: If the dst and src of a certain step in the computation sequence are the same (in-place operation), most AscendC APIs support it, but the specific API needs to be confirmed.

3.2 Tiling strategy

Important: AscendC operators use a two-level Tiling strategy. Refer to the corresponding documentation based on the operator type:

┌─────────────────────────────────────────────────────────────┐
│                    全局内存 (GM)                              │
│  ┌─────────────────────────────────────────────────────┐   │
│  │              totalLength 元素数据                     │   │
│  └─────────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────┘
                           │
          ┌────────────────┼────────────────┐
          ▼                ▼                ▼
    ┌──────────┐     ┌──────────┐     ┌──────────┐
    │  Core 0  │     │  Core 1  │ ... │ Core 39  │   ← Block级Tiling (核间切分)
    └──────────┘     └──────────┘     └──────────┘
          │                │                │
          ▼                ▼                ▼
    ┌──────────┐     ┌──────────┐     ┌──────────┐
    │   UB 0   │     │   UB 1   │     │  UB 39   │   ← UB级Tiling (核内切分)
    └──────────┘     └──────────┘     └──────────┘

Block-level Tiling (inter-core partitioning):

  • Distribute data to multiple AI Cores for parallel processing
  • Load balancing: Full-core/tail-core strategy, the amount of data processed by the tail core is less than or equal to that of a full core

UB-level Tiling (intra-core partitioning):

  • Each Core internally processes data in blocks
  • UB alignment: 32 bytes
  • UB capacity: Does not exceed UB_SIZE_LIMIT (obtained through an interface during actual coding, example value 192KB)

Reference documentation:

  • Element-wise operations: Read references/elementwise-tiling.md(includes a complete two-level Tiling implementation)
  • Reduction operations: Read references/reduction-tiling.md
  • Index operations: Read references/index-tiling.md
  • Sorting operations: Read references/sort-tiling.md
  • Pooling operations: Read references/pooling-tiling.md
  • General principles: Refer to references/general-tiling-principles.md

3.3 Hardware constraints

  • UB buffer: Must be aligned to 32 bytes, even if logically only a small amount of data needs to be stored
  • Reduction-type operators: A single-value buffer needs to allocate 32B of space
  • Precision handling:
    • FP32 input: No precision promotion needed, compute directly
    • FP16 input: Must promote precision to FP32 for computation, to ensure computation precision
    • BF16 input: Must promote precision to FP32 for computation, the vector computation unit does not support direct bfloat16 computation
  • Workspace requirements:
    • elementwise type: SYSTEM_WORKSPACE_SIZE (usually 16MB)
    • Other types of operators: Calculated based on the actual tiling data size

3.4 UB allocation quick reference table for common operator types

Based on the number of operator inputs and the data type, quickly determine bufferCoefficient:

Single-input single-output elementwise (acosh, relu, sigmoid, exp, ln, sqrt, abs...)：

Data type UB layout bufferCoefficient
float32 inQ(2×4) + outQ(2×4) + tmpBuf(1×4) = 20 20
float16 inQ(2×2) + outQ(2×2) + tmpBuf1(1×4) + tmpBuf2(1×4) = 16 16

Dual-input single-output elementwise (add, mul, sub, div...)：

Data type UB layout bufferCoefficient
float32 inQ_X(2×4) + inQ_Y(2×4) + outQ(2×4) + tmpBuf(2×4) = 32 32
float16 inQ_X(2×2) + inQ_Y(2×2) + outQ(2×2) + tmpBuf(3×4) = 24 24

Practical experience: bufferCoefficient is the most critical parameter in the code-gen stage. In the design document, must explicitly provide the value for each dtype; otherwise, code generation cannot correctly compute tileLength.

3.5 Generate design document

Based on the collected information, read the templates/design-template.md template, fill in all sections, and output to csrc/ops/<op_name>/design.md。

Output location: ascend-kernel/csrc/ops/<op_name>/design.md(overwrite the placeholder file from the initialization stage)

Interaction flow

When called by the scheduling skill(recommended flow):

  1. Receive the operator name and function description
  2. Automatically select the implementation path
  3. Read the reference document and generate the complete design document
  4. Output to design.md

When called independently：

  1. Requirements gathering: Understand operator requirements through dialogue
  2. Solution recommendation: Recommend an implementation path based on the requirements
  3. Detailed design: Generate the complete design document
  4. Check and confirm: Confirm key design points with the user
  5. Hand over to development: Generate a checklist and prepare to enter the coding stage

Notes

Tiling parameter design principles

  1. Parameter structuring:

    // 好的做法：使用结构体
struct MyOperatorTilingData {
    int64_t totalLength;        // 总数据长度

    int64_t formerNum;          // 整核数量
    int64_t formerLength;       // 整核数据长度
    int64_t tailNum;            // 尾核数量
    int64_t tailLength;         // 尾核数据长度

    int64_t tileLength;         // UB单次处理长度
};

// 避免：使用大量独立参数
void KernelFunc(int64_t totalLength, int64_t tileNum, int64_t tileLength, ...);

  2. Two-level alignment:

    // Block级: Cache Line 对齐 (512字节)
constexpr int64_t CACHE_LINE_BYTE_LENGTH = 512;
int64_t totalLengthCoreAlign = ((totalLengthCore + CACHE_LINE_BYTE_LENGTH - 1) / CACHE_LINE_BYTE_LENGTH) * CACHE_LINE_BYTE_LENGTH;

// UB级: 32字节对齐
int64_t ubAlignElements = 32 / dtypeSize;
int64_t tileLengthAligned = ((tileLength + ubAlignElements - 1) / ubAlignElements) * ubAlignElements;

  3. UB allocation table: For each operator design,mustinclude a UB allocation table that clearly lists:

    • All buffer names and purposes
    • The size of each buffer (bytes)
    • Number of buffers (single buffer or double buffer)
    • Total UB usage and constraint verification

  4. Double Buffer: Use BUFFER_NUM=2 to implement double buffer and hide memory latency

Other notes

  1. Data type alignment: Ensure PyTorch tensor types match AscendC kernel types (half ↔ float16, float ↔ float32)
  2. Memory alignment: AscendC requires memory address alignment (UB 32B, Cache Line 512B)
  3. Shape constraints: Some operators have special requirements for shape (for example, it needs to be divisible by the tile size)
  4. Performance trade-offs: Find a balance between code complexity and performance
  5. Interface definition: Check whether libraries such as PyTorch/Numpy have similar operator interfaces; if so, refer to PyTorch/Numpy and other libraries for the interface definition
  6. Test input range: Note the valid input range of the operator in the design document (for example, acosh requires x >= 1); test cases must generate data accordingly

Definition of Done (DoD)

After the design document is generated, it must include the following key deliverables (for direct consumption by the code-gen skill):

  • Function signature: at::Tensor op_name(const at::Tensor &self, ...) complete declaration
  • Supported data types: List explicitly (such as float16, float32)
  • AscendC API call pseudocode: Map each computation step to a specific API
  • UB allocation table: Buffer layout and bufferCoefficient for each dtype
  • Tiling parameter struct: Field definitions and calculation formulas
  • FP16/BF16 precision promotion process: If half precision is supported, the Cast path must be described

Next steps

After the design is complete, use ascendc-operator-code-gen skill to generate the specific code implementation.