SuperAPI AI Directory
Partner with us中文AI newsFavorites
Skillsexternal-gitcode-ascend-ascendc-operator-design

external-gitcode-ascend-ascendc-operator-design

Design & creation

View repository

Updated 10-15 17:19

About this skill

AscendC Operator Design Skill

Generate a complete design document (design.md) based on operator requirements, for consumption by the subsequent code-gen skill.

Usage Scenarios

At ascendc-operator-project-init after the skeleton is created andascendc-operator-code-gen before code is generated.

Design Process

1. Operator Requirement Analysis

If invoked by the orchestration skill, the operator name and function description are already determined; proceed directly to step 2.

If invoked independently, confirm the following information with the user:

Information Required Description
Operator name (snake_case) Yes such as acosh, rms_norm
Function description / mathematical formula Yes such as "acosh(x) = ln(x + sqrt(x²-1))"
Supported data types No default float16 + float32

MANDATORY: Check whether PyTorch / NumPy has an interface with the same name. If it exists, the interface signature and semantics must be aligned with it (such as torch.acosh、torch.softmax）。

2. Choose the Implementation Path

Based on the operator's characteristics, recommend a suitable implementation approach:

Implementation Path Applicable Scenario Judgment Criterion
AscendC Kernel Pure vector operator Does not involve matrix multiplication
CATLASS template library GEMM / FlashAttention Contains cube matrix computation
ACLNN wrapper CANN already has a built-in operator No custom kernel needed

New operators use the AscendC Kernel path by default, unless matrix multiplication is explicitly involved.

3. Detailed Design Document Generation

MANDATORY: Before generating the design document, the following reference documents must be read:

  1. Required reading: templates/design-template.md — design document template
  2. Optional reading by operator type:
    • Element-wise operations (add/relu/acosh/sigmoid...) → references/elementwise-tiling.md
    • Reduction operations (softmax/layernorm...) → references/reduction-tiling.md
  3. General reference: references/general-tiling-principles.md

Never skip reading the reference documents.

3.1 Design Document Structure

The design document contains the following core sections:

  1. Operator Interface Definition — function signature, parameter description, supported data types
  2. Computation Logic Design — algorithm description,AscendC API call pseudocode, implementation path selection
  3. Tiling Strategy — two-level Tiling design (Block level + UB level), UB allocation table, tileLength calculation
  4. Workspace Requirements — workspace size calculation
  5. Performance Optimization — key optimization points, operator characteristic analysis
  6. Kernel-Side Implementation Points — offset calculation, execution flow,FP16/BF16 precision promotion flow
  7. Implementation Checklist — file structure, code points, test points

3.2 Pseudocode for Computation Logic (Key Deliverable)

Must decompose mathematical formulas into AscendC API call sequences. This is the direct input to the code-gen skill.

Mapping of Common Mathematical Functions to AscendC APIs：

Mathematical Operation AscendC API Remarks
x + y Add(dst, src0, src1, len) Dual input
x - y Sub(dst, src0, src1, len) Dual input
x * y Mul(dst, src0, src1, len) Dual input
x / y Div(dst, src0, src1, len) Dual input
x + scalar Adds(dst, src, scalar, len) Scalar operation, preferred
x * scalar Muls(dst, src, scalar, len) Scalar operation, preferred
abs(x) Abs(dst, src, len)
exp(x) Exp(dst, src, len)
ln(x) Ln(dst, src, len)
sqrt(x) Sqrt(dst, src, len)
1/x Reciprocal(dst, src, len)
1/sqrt(x) Rsqrt(dst, src, len)
tanh(x) Tanh(dst, src, len)
relu(x) Relu(dst, src, len)
max(x,y) Max(dst, src0, src1, len)
min(x,y) Min(dst, src0, src1, len)
fp16→fp32 Cast(dst, src, CAST_NONE, len) Precision-increasing, lossless
fp32→fp16 Cast(dst, src, CAST_ROUND, len) Precision-decreasing, lossy

Example — API call sequence for acosh(x)：

Mul(tmp, x, x, len);        // tmp = x²
Adds(tmp, tmp, -1.0f, len); // tmp = x² - 1
Sqrt(tmp, tmp, len);         // tmp = sqrt(x² - 1)
Add(tmp, tmp, x, len);      // tmp = x + sqrt(x² - 1)
Ln(y, tmp, len);             // y = ln(x + sqrt(x² - 1))

Note: If a step's dst in the computation sequence is the same as src (in-place operation), most AscendC APIs support it, but the specific API must be confirmed.

3.2 Tiling Strategy

Important: AscendC operators use a two-level tiling strategy. Refer to the corresponding documentation based on the operator type:

┌─────────────────────────────────────────────────────────────┐
│                    全局内存 (GM)                              │
│  ┌─────────────────────────────────────────────────────┐   │
│  │              totalLength 元素数据                     │   │
│  └─────────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────┘
                           │
          ┌────────────────┼────────────────┐
          ▼                ▼                ▼
    ┌──────────┐     ┌──────────┐     ┌──────────┐
    │  Core 0  │     │  Core 1  │ ... │ Core 39  │   ← Block级Tiling (核间切分)
    └──────────┘     └──────────┘     └──────────┘
          │                │                │
          ▼                ▼                ▼
    ┌──────────┐     ┌──────────┐     ┌──────────┐
    │   UB 0   │     │   UB 1   │     │  UB 39   │   ← UB级Tiling (核内切分)
    └──────────┘     └──────────┘     └──────────┘

Block-level Tiling (Inter-core Partitioning):

  • Distribute data to multiple AI Cores for parallel processing
  • Load balancing: Full-core/tail-core strategy, the tail core processes an amount of data less than or equal to that of a full core

UB-level Tiling (Intra-core Partitioning):

  • Process data in chunks within each Core
  • UB alignment: 32 bytes
  • UB capacity: Does not exceed UB_SIZE_LIMIT (obtained via the interface during actual coding; example value 192KB)

Reference Documentation:

  • Element-wise operations: Read references/elementwise-tiling.md(includes the complete two-level tiling implementation)
  • Reduction operations: Read references/reduction-tiling.md
  • General principles: Refer to references/general-tiling-principles.md

3.3 Hardware Constraints

  • UB buffer: Must be aligned to 32 bytes, even if only a small amount of data needs to be stored logically
  • Reduction-type operators: Single-value buffers need to allocate 32B of space
  • Precision handling:
    • FP32 input: No precision increase needed, compute directly
    • FP16 input: Must increase precision to FP32 for computation, to ensure computational precision
    • BF16 input: Must increase precision to FP32 for computation, the vector compute unit does not support direct bfloat16 computation
  • Workspace requirements:
    • elementwise type: SYSTEM_WORKSPACE_SIZE (typically 16MB)
    • Other operator types: Calculated based on the actual tiling data size

3.4 UB Allocation Quick Reference for Common Operator Types

Based on the number of operator inputs and the data type, quickly determine bufferCoefficient:

Single-input single-output elementwise (acosh, relu, sigmoid, exp, ln, sqrt, abs...)：

Data type UB layout bufferCoefficient
float32 inQ(2×4) + outQ(2×4) + tmpBuf(1×4) = 20 20
float16 inQ(2×2) + outQ(2×2) + tmpBuf1(1×4) + tmpBuf2(1×4) = 16 16

Dual-input single-output elementwise (add, mul, sub, div...)：

Data type UB layout bufferCoefficient
float32 inQ_X(2×4) + inQ_Y(2×4) + outQ(2×4) + tmpBuf(2×4) = 32 32
float16 inQ_X(2×2) + inQ_Y(2×2) + outQ(2×2) + tmpBuf(3×4) = 24 24

Practical experience: bufferCoefficient is the most critical parameter in the code-gen phase. In the design document, Must Explicitly provide the value for each dtype; otherwise, code generation cannot correctly calculate tileLength.

3.5 Generate Design Document

Based on the collected information, read the templates/design-template.md template, fill in all sections, and output to csrc/ops/<op_name>/design.md。

Output location: ascend-kernel/csrc/ops/<op_name>/design.md(overwrite the placeholder file from the initialization phase)

Interaction Flow

When invoked by the scheduling skill(recommended flow):

  1. Receive the operator name and function description
  2. Automatically select the implementation path
  3. Read the reference document and generate the complete design document
  4. Output to design.md

When invoked independently：

  1. Requirements collection: Understand the operator requirements through dialogue
  2. Solution recommendation: Recommend an implementation path based on the requirements
  3. Detailed design: Generate the complete design document
  4. Review and confirmation: Confirm the key design points with the user
  5. Handoff to development: Generate a checklist and prepare to enter the coding phase

Precautions

Tiling Parameter Design Principles

  1. Structured parameters:

    // 好的做法：使用结构体
struct MyOperatorTilingData {
    int64_t totalLength;        // 总数据长度

    int64_t formerNum;          // 整核数量
    int64_t formerLength;       // 整核数据长度
    int64_t tailNum;            // 尾核数量
    int64_t tailLength;         // 尾核数据长度

    int64_t tileLength;         // UB单次处理长度
};

// 避免：使用大量独立参数
void KernelFunc(int64_t totalLength, int64_t tileNum, int64_t tileLength, ...);

  2. Two-level alignment:

    // Block级: Cache Line 对齐 (512字节)
constexpr int64_t CACHE_LINE_BYTE_LENGTH = 512;
int64_t totalLengthCoreAlign = ((totalLengthCore + CACHE_LINE_BYTE_LENGTH - 1) / CACHE_LINE_BYTE_LENGTH) * CACHE_LINE_BYTE_LENGTH;

// UB级: 32字节对齐
int64_t ubAlignElements = 32 / dtypeSize;
int64_t tileLengthAligned = ((tileLength + ubAlignElements - 1) / ubAlignElements) * ubAlignElements;

  3. UB allocation table: Each operator designmustinclude a UB allocation table that clearly lists:

    • Names and purposes of all buffers
    • Size of each buffer (bytes)
    • Number of buffers (single buffer or double buffer)
    • Total UB usage and constraint verification

  4. Double Buffer: Use BUFFER_NUM=2 to implement double buffer and hide memory latency

Other Precautions

  1. Data type alignment: Ensure that the PyTorch tensor type and the AscendC kernel type match (half ↔ float16, float ↔ float32)
  2. Memory alignment: AscendC requires memory address alignment (UB 32B, Cache Line 512B)
  3. Shape constraints: Some operators have special requirements for shape (for example, it needs to be divisible by tile size)
  4. Performance trade-offs: Find a balance between code complexity and performance
  5. Interface definition: Check whether libraries such as PyTorch/Numpy have similar operator interfaces. If they exist, refer to PyTorch/Numpy and other libraries for the interface definition
  6. Test input range: Specify the valid input range of the operator in the design document (for example, acosh requires x >= 1), and test cases must generate data accordingly

Definition of Done (DoD)

After the design document is generated, it must include the following key deliverables (for direct consumption by the code-gen skill):

  • Function signature: at::Tensor op_name(const at::Tensor &self, ...) Complete declaration
  • Supported data types: Clearly list them (such as float16, float32)
  • AscendC API call pseudocode: Each calculation step is mapped to a specific API
  • UB allocation table: The buffer layout and bufferCoefficient for each dtype
  • Tiling parameter struct: Field definitions and calculation formulas
  • FP16/BF16 precision promotion flow: If half precision is supported, the Cast path must be described

Next Steps

After the design is completed, use ascendc-operator-code-gen skill Generate specific code implementation.