SuperAPI AI Directory
Partner with us中文AI newsFavorites
Skillsexternal-cannbot-ops-pypto-golden-generate

external-cannbot-ops-pypto-golden-generate

Coding & development

View repository

Updated 10-15 17:19

About this skill

PyPTO Golden Reference Implementation Generation

Based on the operator specification information, automatically generate a PyTorch golden reference implementation and complete verification code. The generated golden script is used for quickly verifying the correctness of the operator implementation during the development phase, and can be imported and called as an independent module by test_{op}.py and other scripts. Generated based on the fixed template templates/golden-template.py (read this template in the §9 generate file structure phase). Pure torch implementation, introducing pypto is prohibited.

  1. Extract the operator name, formula, input/output specifications and other necessary information from the user input
  2. If the information is insufficient, ask the user step by step to supplement it
  3. Execute golden function generation and verification according to the workflow
  4. Output {op}_golden.py to the current directory or a user-specified location

1. Required Information

Item Description
Input Operator specification information (such as structured specification content, natural language description, etc.)
Output {op}_golden.py, export the {op}_golden() function, path determined by the caller

2. Operator Information Acquisition

Extract the operator name from the input, or have the caller specify it. If the information is insufficient, ask the user step by step to supplement it.

3. Specification Field Check

After reading the operator specification information, check field completeness according to the following categories:

Required fields (report an error and exit if missing)

Field Location Purpose
Operator name §1 Basic Information File name, function name
Mathematical formula §1 Basic Information Generate the PyTorch implementation
Input specification §4 Data Specification Function parameters, validate shape
Output specification §4 Data Specification Return type, validate shape

Recommended fields (guide the user to supplement if missing)

Field Location Purpose Handling when missing
Typical configuration §11 Application Scenarios Typical case verification Guide the user to supplement it into the specification information
Dynamic axis range §7 Dynamic Axis Description Generalization case sampling Use the default range (1-1024)

Handling when the typical configuration is missing

检查规格信息中是否包含典型配置
    │
    ├── 有 → 直接使用
    │
    └── 无 → 引导用户提供
            │
            ├── 用户提供 → 补充到规格信息中，继续生成
            │
            └── 用户跳过 → 根据动态轴范围生成默认配置
                          │
                          ├── 有动态轴范围 → 按范围推荐
                          │   └── 补充到规格信息中，继续生成
                          └── 无动态轴范围 → 使用通用默认值
                              └── 补充到规格信息中，继续生成

The typical configuration uses a 7-column format:

Configuration name Type Priority Parameter Input Shape Output Shape Description

4. Golden Function Generation Specifications

Implementation Method

Use PyTorch to implement the golden function. Prefer to use PyTorch built-in APIs; when there is no directly corresponding API, have the LLM generate the implementation based on the formula.

Function Signature

# 单输入
def silu_golden(x: torch.Tensor) -> torch.Tensor:

# 多输入
def swiglu_golden(x: torch.Tensor, gate: torch.Tensor) -> torch.Tensor:

# 带可选参数
def layer_norm_golden(
    x: torch.Tensor,
    normalized_shape: List[int],
    weight: Optional[torch.Tensor] = None,
    bias: Optional[torch.Tensor] = None,
    eps: float = 1e-5,
) -> torch.Tensor:

Rule：

  • Function name:{算子名}_golden
  • Parameter order: required tensor parameters first, optional parameters after
  • All parameters defined in the spec must be implemented, including optional parameters
  • dtype is not treated as a parameter; computation precision follows the input tensor

Boundary condition mapping

Generate the corresponding code logic based on the boundary condition definitions in the specification information:

# spec 定义：零值返回 1
def safe_div_golden(x: torch.Tensor, y: torch.Tensor) -> torch.Tensor:
    result = x / y
    result[y == 0] = 1.0  # 映射边界条件
    return result

5. Confidence system

Evaluate confidence based on the implementation approach, affecting the annotations and prompt strength of the final output:

1. 是否有等效 PyTorch API？
   ├─ 有 → 直接调用 → ⭐⭐⭐⭐⭐
   │
   └─ 无 → 2. 是否为已知论文算子？
            ├─ 是 → 搜索第三方实现 → ⭐⭐⭐⭐
            │
            └─ 否 → 3. LLM 智能转换
                     ├─ 简单公式转换 → ⭐⭐⭐⭐
                     └─ 复杂公式转换 → ⭐⭐⭐

Strengthen prompts for low confidence(⭐⭐⭐ and below):

Add a prominent warning in the docstring of the output file, reminding to manually review the code logic, verify against the paper, and validate boundary cases using multiple sets of data.

Confidence and validation are independent mechanisms—confidence only affects annotations and prompt strength, while validation standards are consistent for all operators. After automatic fixes, update confidence based on the final code.

6. Validation mechanism

Validation execution method

After generating the file,validation must be completed by directly executing the script：

python3 {op}_golden.py

The script contains a if __name__ == "__main__": _validate() entry, which will automatically run all check items (typical cases, generalization cases, value range, numerical stability, etc.) and output a validation report.

Prohibitedusing the following methods as substitutes for direct execution:

  • exec(open(...).read()) — an independent execution environment that bypasses the script
  • Manually constructing test data to call the golden function separately — duplicates the script's built-in validation logic and is incomplete

Gate determination basis: based on the python3 {op}_golden.py 's exit code (0 = pass) and the validation report output.

Validation shape sources

Two-layer sources：

├── 典型 case（来自算子规格中的典型配置）
│   ├── 性能类型：按优先级验证，P0 必须通过
│   └── 功能类型：按优先级验证，P0 必须通过
│
└── 泛化 case（来自算子规格中的动态轴取值范围）
    └── 每个动态轴采样：最小值、中间值、最大值
        如 b: 1-128 → [1, 64, 128]
           s: 64-2048 → [64, 1024, 2048]
        组合生成 shape

Validation order (by importance)

  1. Performance type P0 configuration (highest priority, must pass)
  2. Performance type P1 configuration
  3. Functional type P0 configuration (must pass)
  4. Functional type P1 configuration
  5. Generalization cases (boundary + intermediate sampling)
  6. Other low-priority configurations

Validation check items

Check item Description Level
Syntax check Code can be imported normally 🔴 Critical
Shape consistency Output shape matches the spec definition 🔴 Critical
Function signature Parameters match the spec definition 🔴 Critical
Value range check Derive value range constraints from the formula 🟡 Warning
Special point validation Boundary values, zero values, etc. 🟡 Warning
Numerical stability No NaN/Inf 🟡 Warning
Mathematical properties Monotonicity, symmetry, conserved quantities, etc. 🟡 Warning
API comparison Compare with the PyTorch API (if applicable) 🟢 Info

Validation properties derived from the formula

Value range constraint: Derive the output range based on the formula's properties. For example, sigmoid's output is in (0, 1), ReLU's output is >= 0, and softmax's output is in (0, 1) and sums to 1 along the reduction axis.

Mathematical properties: Check parity (tanh is an odd function), monotonicity (sigmoid is monotonically increasing), conserved quantities (softmax sums to 1), etc.

Special point verification: Verify the output for key input values, such as sigmoid(0) = 0.5, tanh(0) = 0, relu(0) = 0.

7. Automatic repair strategy

When verification fails, automatically attempt repairs in priority order:

  1. Use stable PyTorch APIs — replace hand-written implementations with PyTorch built-in APIs

    • x / (1 + torch.exp(-x)) → torch.sigmoid(x)
    • torch.exp(x) / torch.exp(x).sum() → torch.softmax(x, dim=-1)

  2. Add numerical stability protection — prevent division by zero, log of negative numbers, etc.

    • a / b → a / (b + 1e-8)
    • torch.log(x) → torch.log(torch.clamp(x, min=1e-8))

  3. Use a more stable equivalent form — a mathematically equivalent but numerically more stable alternative

Limit: at most 3 repair attempts. After repair, update the confidence based on the final code.

8. Error grading and output control

Level Meaning Handling after repair failure
🔴 Severe Syntax errors, shape mismatches, signature errors Block output, report the reason for the error
🟡 Warning Range check failure, numerical instability Output file + annotate the warning in the docstring
🟢 Pass All checks passed Normal output

9. Generated file structure

The generated file follows a fixed template templates/golden-template.py, and during generation the template is read and the placeholders are replaced with the actual operator content. The template contains the following structure:

  • File-level docstring (operator name, formula, confidence)
  • {op}_golden() function: pure PyTorch reference implementation (with example comments)
  • _validate() function: automatic verification (typical case, generalization case, range check, numerical stability, API comparison)
  • if __name__ == "__main__": _validate() entry point

10. User interaction

Fully automatic generation and verification, with user involvement required only in the following scenarios:

Scenario Interaction method
Generation, verification, repair Fully automatic, no user involvement required
Typical configuration missing Guide the user to provide or confirm the recommended configuration
Overwrite existing file Via AskUserQuestion ask for confirmation

11. Exception handling

Scenario Handling method
Missing operator specification information Error exit, prompting the user to first fill in the required information
Failed to parse specification information Error exit, prompting that the input format is incorrect
Specification information missing required fields List the missing fields and guide the user to supplement them
Multiple operators unspecified List all available operators and require the user to specify one
golden file already exists Via AskUserQuestion ask whether to overwrite

12. Verification report (must be executed)

After the file generation is complete, display the verification result to the user, for example:

✅ Golden 参考实现已生成:
  • {name}_golden.py

============================================================
attention_golden 验证报告
============================================================

[典型 case 验证]
  性能_P0: b=2,h=8,s=512,d=64,w=128 ... ✓ PASS
  功能_P0: b=1,h=4,s=256,d=64,w=128 ... ✓ PASS

[泛化 case 验证]
  b=1,h=4,s=128,d=32,w=64 ... ✓ PASS
  b=64,h=8,s=1024,d=64,w=128 ... ✓ PASS

[值域检查]
  检查 softmax 归一化 ... ✓ PASS

[数值稳定性检查]
  大值输入 (x=100) ... ✓ PASS
  小窗口 (w=4) ... ✓ PASS

[功能正确性检查]
  验证窗口边界 ... ✓ PASS

============================================================
✅ 所有验证通过
============================================================