ascendc-operator-compile-debug

name: ascendc-operator-compile-debug description: Compile and install AscendC operators and run precision tests. TRIGGER when: after operator code generation is complete and compilation verification, whl package installation, or precision test execution is needed, or when compilation/testing fails and troubleshooting is required. Keywords: build.sh, compile, install, whl, pytest, precision test, compilation error, NPU test.

AscendC Operator Compilation, Installation, and Precision Testing

Compile the ascend-kernel project, install the whl package, generate and run precision tests. Usually invoked by the ascendc-operator-code-gen skill after code generation is complete.

Prerequisites

op_host, op_kernel code has been generated
ops.h, register.cpp, csrc/CMakeLists.txt have been updated (framework adaptation complete)
CANN environment is available

Workflow

Phase 0: Environment Preparation

MANDATORY — The environment must be loaded before every shell command. Hardcoding paths is prohibited; searching for paths on your own is prohibited.

Obtain Environment Information

If an upstream skill (such as ascendc-operator-dev) has already confirmed the CANN path and Conda environment name, use them directly without asking again.

If environment information was not obtained from upstream, confirm it using the following process:

CANN Environment：

Check the environment variable ASCEND_HOME_PATH（echo $ASCEND_HOME_PATH）
If already set: use it directly, treating it as the CANN_PATH
If not set：MUST ask the user for the CANN installation path

Conda Environment：

Check whether a conda environment is currently activated (echo $CONDA_DEFAULT_ENV）
If already activated(value is not base and not empty): use the current environment directly
If not activated or is base：MUST ask the user for the conda environment name

Activate the Environment

After confirmation, load it before every shell command:

source ${CANN_PATH}/*/set_env.sh
conda activate <env_name>

Practical experience: Not loading the environment will cause: (1) build.sh cannot find the AscendC compiler; (2) pip/python versions are incorrect; (3) torch_npu cannot be found when running tests. It is recommended to source it once before every shell command.

Phase 1: Compile

cd ascend-kernel
chmod +x build.sh  # 模板复制后可能缺少执行权限
bash build.sh

Determining Compilation Success:

ls ./output/ascend_kernel*.whl

If the .whl file exists, it is successful.

Compilation failure: enter the troubleshooting loop (see below), up to 3 times.

Phase 2: Install

cd ascend-kernel
pip install output/ascend_kernel*.whl --force-reinstall --no-deps

Phase 3: Generate Test Files

First check whether ascend-kernel/tests/test_<op_name>.py already exists; if it exists, skip generation.

If it does not exist, under ascend-kernel/tests/ create test_<op_name>.py。

Test File Structure:

#!/usr/bin/env python3
"""Unit tests for <op_name> NPU operator."""

import torch
import torch_npu
import pytest

try:
    import ascend_kernel
except ImportError:
    import os, glob
    script_dir = os.path.dirname(os.path.abspath(__file__))
    project_root = os.path.dirname(script_dir)
    lib_pattern = os.path.join(project_root, "python/ascend_kernel/ascend_kernel/lib/*.so")
    lib_files = glob.glob(lib_pattern)
    if lib_files:
        torch.ops.load_library(lib_files[0])
    else:
        raise ImportError("Could not find ascend_kernel library")


def is_npu_available():
    try:
        return torch.npu.is_available()
    except Exception:
        return False


@pytest.fixture(scope="module")
def device():
    if not is_npu_available():
        pytest.skip("NPU not available")
    return torch.device("npu:0")


class Test<OpName>:

    @pytest.mark.parametrize("shape", [
        # 根据算子特点选择典型 shape
    ])
    @pytest.mark.parametrize("dtype", [torch.float16, torch.float32])
    def test_basic(self, device, shape, dtype):
        x = torch.randn(shape, dtype=dtype, device=device)

        npu_result = torch.ops.npu.<op_name>(x)
        cpu_reference = <pytorch_reference>(x.cpu())

        rtol, atol = (1e-3, 1e-3) if dtype == torch.float16 else (1e-5, 1e-5)
        assert torch.allclose(npu_result.cpu(), cpu_reference, rtol=rtol, atol=atol), \
            f"max diff = {(npu_result.cpu() - cpu_reference).abs().max().item()}"


def run_simple_test():
    """简单功能测试（无精度校验）"""
    if not is_npu_available():
        print("NPU not available, skipping test")
        return False

    device = torch.device("npu:0")
    try:
        for dtype in [torch.float16, torch.float32]:
            x = torch.randn(1024, dtype=dtype, device=device)
            result = torch.ops.npu.<op_name>(x)
            print(f"dtype={dtype}, output shape={result.shape} OK")
        print("Test PASSED!")
        return True
    except Exception as e:
        print(f"Test FAILED: {e}")
        import traceback
        traceback.print_exc()
        return False


if __name__ == "__main__":
    run_simple_test()

Key Points for Test Case Design:

Scenario	Parameterization Content	Precision Standard
Basic shape	1D/2D/3D/4D	fp32: rtol=1e-5, fp16/bf16: rtol=1e-3
Different sizes	1, 7, 256, 65536, 1M	Same as above
Boundary values	All zeros, all negative, extremely large values	Absolute value comparison
Operator-specific	According to the constraints in the design document	According to operator characteristics

CPU Reference Implementation: Use native PyTorch functions (such as torch.acosh、torch.softmax , etc.). If there is no direct equivalent, write the Python reference logic by hand.

Precision Comparison Strategy:

FP32 NPU result vs FP32 CPU result: compare directly
FP16 NPU result vs CPU reference: first compute on CPU with FP32, then .to(dtype) convert back and compare

Test Data Generation Notes:

Check the operator's valid input range (e.g., acosh requires x >= 1, log requires x > 0, sqrt requires x >= 0)
When generating data, use torch.rand() * scale + offset to ensure it falls within the valid range
Include boundary value tests (e.g., acosh(1)=0, relu(0)=0)

Phase 4: Run Tests

Functional Tests(must pass):

cd ascend-kernel && python tests/test_<op_name>.py

Precision Tests(recommended):

cd ascend-kernel && pytest tests/test_<op_name>.py -v

Test Pass Criteria:

Functional Tests: Program exits normally (exit code 0)
Precision Tests: All pytest cases pass

Debugging Loop (up to 3 times)

失败 → 分析错误日志 → 定位错误文件:行号 → 修复代码 → 重新编译/测试
  ↓ (仍失败)
第 2 次 → 重新分析，检查遗漏的 include/声明/CMakeLists
  ↓ (仍失败)
第 3 次 → 深入检查 API 用法、类型匹配、CMake 配置
  ↓ (仍失败)
停止 → 向用户报告详细错误信息和已尝试的修复

Compilation Error Decision Tree

Error Symptom	Location to Modify	Troubleshooting Direction
`undeclared identifier`	`ops.h` or op_host cpp	Missing function declaration or include
`no matching function`	op_host cpp	Parameter type/order does not match the kernel entry point
`undefined reference` / linker error	`csrc/CMakeLists.txt`	Source file not added to OP_SRCS or ascendc_library
`redefinition`	ops.h / register.cpp	Duplicate definition
AscendC kernel compilation error	op_kernel cpp	API usage or type not supported

Test Error Decision Tree

Error Symptom	Troubleshooting Direction
`ImportError: ascend_kernel`	whl not installed or so not generated
`RuntimeError: ... not found`	register.cpp registration name does not match the call name
`allclose failed`	Computation logic or precision issue, check the kernel Compute function
`shape mismatch`	op_host output tensor shape calculation is incorrect
`SIGABRT / device error`	tiling parameter error or UB out-of-bounds

Anti-Pattern Checklist

NEVER Hardcoding CANN paths or Conda environment paths, must obtain from upstream or ask the user
NEVER Searching the file system on your own to guess CANN or Conda paths
NEVER Modifying cmake/ or csrc/utils/ files under
NEVER Deleting other operators' registration code during troubleshooting
NEVER Skipping functional tests and going straight to precision tests
NEVER Ignoring compilation warnings (especially those related to type truncation)
NEVER Passing rvalues (temporary objects, literals, expressions) in EXEC_KERNEL_CMD
NEVER Continuing to try after more than 3 failed troubleshooting attempts, should report to the user

Modifiable File Scope

File	Modification Content
`csrc/ops/<op_name>/op_host/*.cpp`	host-side logic
`csrc/ops/<op_name>/op_kernel/*.cpp`	kernel-side logic
`csrc/ops.h`	Function declaration
`csrc/register.cpp`	Operator registration
`csrc/CMakeLists.txt`	Compilation source file list
`tests/test_<op_name>.py`	Test files