SuperAPI AI Directory
Partner with us中文AI newsFavorites
Skillsascend-opplugin

ascend-opplugin

Coding & development

View repository

Updated 10-15 17:19

About this skill

ascend-opplugin

This skill focuses on Ascend C operators(such as op_host / op_kernel, tiling, CANN build chain) once the implementation is ready, how tointegrate it into torch_npu and make it usable in PyTorch: file placement within the project, Host registration, compilation and packaging, installation and verification.Prioritize minimal intrusion and reproducible builds; if the user already has an operator repository, by default complete the integration within their project; only reference or use the op-plugin repository when needed. Subsequent sections of the main text will elaborate on environment and step details by scenario.

Overview of Integration Approaches

From the PyTorch side, custom NPU operators commonly have 4 integration routes:

  • torch.library / TORCH_LIBRARY[_IMPL]：
    • This is the main route for formally integrating into the PyTorch operator system, suitable for scenarios that require schema,torch.ops.* invocation, and long-term maintenance
    • Formal operator integration defaults to primarily using this route
  • Pybind：
    • Suitable for quickly exposing C++ functions to Python for joint debugging
    • But it is not a complete PyTorch operator registration route and is not suitable as the default solution for formal operator integration
    • It can serve as an auxiliary route when quick verification is needed; for detailed instructions see references/pybind.md
  • op-plugin / OpCommand：
    • Suitable for scenarios where the underlying graph operator already exists and only a PyTorch host wrapper layer needs to be added
    • Suitable for scenarios where the underlying operator already exists and only a PyTorch Host wrapper needs to be added
  • Graph mode / graph integration route (such as TorchAir)：
    • Suitable for scenarios that explicitly need to support graph mode compilation and graph optimization
    • The current skill only provides basic reminders and does not treat it as the default mainline

Selection Recommendations

  • The goal is formal operator integration: prefer the torch.library route
  • The goal is only quick verification of the kernel or interface joint debugging: you may temporarily consider Pybind
  • There is already an underlying CANN / xpu_kernel graph operator: prefer OpCommand
  • The goal is graph mode, graph integration, compilation optimization: TorchAir/graph mode integration requires additional evaluation and the ordinary eager integration process should not be directly applied
  • If you only want to first verify "whether a C++ function can be called from Python" and do not yet require torch.ops.*, schema, or long-term maintenance, you can see references/pybind.md

Minimal Closed Loop

No matter which route is ultimately chosen, it is recommended to first ensure the minimal closed loop is established:

  1. Environment is usable:torch_npu, CANN, and the NPU runtime environment are normal
  2. Core implementation is clear: whether it is a self-developed kernel or reusing an existing graph operator
  3. Interface exposure is complete: it can be called stably from the Python side
  4. Build artifacts are reproducible: it can be recompiled and reinstalled
  5. Tests can run: there is at least one smoke test verifying that registration and invocation succeed

If the user asks "how to use this skill"

If the user explicitly wants to know "how can I ask questions, how can I trigger this skill", you can provide a few representative prompt examples, preferably selected by scenario from references/prompt-examples.md ; there is no need to show all of them at once.

0. Important Principles (Must Be Followed)

  • Prioritize "integration only"; do not modify operator implementations unless necessary：
    • By default, do not modify directories and files related to the operator's own implementation (e.g. op_host/、op_kernel/、tiling、json、infershape）。
    • Prefer completing exposure and invocation by adding an independent integration layer (framework/ or an independent {op_name}_extension/) (especially when reusing a graph operator / OpCommand ).
    • Only consider changing the operator implementation in the following cases: the user explicitly requests to "implement/fix the operator itself", or the existing operator is unavailable and the integration layer cannot work around it.
  • Integration must be accompanied by an executable verification script：
    • Whether building inside the container or driving an in-container build from outside the container, each integration must provide a script under test/ .
    • Minimum requirements:smoke test(can import/load_library, can call torch.ops.npu.<op>, check output dtype/shape).
    • Recommended: clearly document the test run command in the README; when verifying, try to execute outside the project directory (e.g. /tmp), to avoid importing local source code and causing misjudgment.

1. Quick pre-check and solution selection

  • First decide whether to use Plan A or Plan B:
    • If you already have an operator implementation project locally (e.g. one containing op_host、op_kernel、framework, tests, or build files in a xpu_kernel / custom-ops / AscendC repository), prefer using Plan A。
    • Only use Plan B：
      • the user explicitly requests integration based on op-plugin ;
      • the operator team requires the final artifact to land in the op-plugin directory tree;
      • Plan A cannot satisfy the required packaging, build, or test process.
  • Check whether torch_npu is already available in the current environment:
    • Run:
      • python - << 'EOF'
      • import torch; import torch_npu
      • print("torch:", torch.__version__)
      • print("torch_npu:", getattr(torch_npu, "__version__", "unknown"))
      • print("npu available:", torch.npu.is_available())
      • EOF
    • If the import succeeds and npu available: True, you can usually continue using Plan A, and go directly toSection 4to start operator integration.
  • If torch_npu is missing or unavailable: then consider proceeding to Section 2 of Plan Bto prepare the op-plugin environment.
  • The following operations must usethe same Python environment(the same venv/conda):
    • Build torch_npu
    • Build custom operators
    • Install wheel
    • Run tests

Interaction conventions between the in-container environment and the out-of-container command line

If NPU / CANN / torch_npu the environment is inside the container, while the agent can only be driven from outside the container via the command line, then preferably interact as follows:

  • First confirm 3 pieces of information:

    • Container name or container ID (e.g. the target container seen in docker ps / podman ps )
    • The mapping between the host workspace path and the in-container mount path
    • The path to the Python environment used for building/testing inside the container

  • All pre-checks, builds, installations, and tests are executed inside the container:

    • Do not use the host Python to determine whether torch_npu is available
    • Uniformly change to docker exec <container> bash -lc '<cmd>' or podman exec <container> bash -lc '<cmd>'

  • All generated files must reside in a directory that is a "shared mount between the host and the container":

    • In this way, files created/edited by the agent on the host side can be seen directly inside the container
    • In this way, the artifacts built inside the container build/、dist/, and test logs, can also be read directly on the host side
    • Do not write key integration files to temporary paths inside the container, otherwise they cannot be continuously edited and reused outside the container

  • Every time exec the environment must be re-established:

    • Non-interactive bash -lc will not inherit the temporary environment of the previous shell
    • Therefore, each execution should explicitly include source <CANN>/set_env.sh && <venv 激活命令> && <实际命令>

  • Prefer to merge "multi-step dependent actions" into the same container command:

    • For example source ... && python setup.py bdist_wheel && pip install ... && cd test && python test.py
    • This can reduce errors caused by state switching inside and outside the container

  • Import/registration verification should likewise be performed inside the container, and as far as possible outside the project directory:

    • Avoid the source directory inside the container taking precedence in sys.path, which would cause importing the local source by mistake instead of the installed wheel

  • If any of the container name, mount path mapping, or Python environment is unclear, ask the user first; do not guess. For more on "driving build/verification inside the container from the command line outside the container (including OpCommand and other templates and fixes for typical errors)", see the reference documentation:

  • references/container-cli-driver.md

2. Install the op-plugin environment only under Plan B

This stepis not the default path. Execute it only when it is certain that Plan B is adopted. When executing, also try to keep itidempotent: prefer reusing the existing environment, and rebuild only when it is missing or the version does not match.

  • 2.1 Source the CANN environment once per shell

    • source <CANN install path>/set_env.sh(for example /usr/local/Ascend/ascend-toolkit/set_env.sh）

  • 2.2 Check whether the existing op-plugin repository and torch_npu already match

    • If you already have a op-plugin checkout, and the torch_npu in the same Python environmentskip the rebuild：
      • Example check command:
        • python - << 'EOF'
        • import torch, torch_npu
        • print("torch:", torch.__version__)
        • print("torch_npu:", getattr(torch_npu, "__version__", "unknown"))
        • print("npu available:", torch.npu.is_available())
        • EOF
      • If the import succeeds,npu available: True, and the version meets the project requirements, you can proceed directly toSection 4for operator integration.

  • 2.3 If torch_npuis missing locally, or a specific version is required, prepare/switch the op-plugin environment

    • Clone op-plugin(the branch must match the target torch_npu version):
      • git clone --branch 7.3.0 https://gitcode.com/ascend/op-plugin.git && cd op-plugin
    • Build:
      • bash ci/build.sh --python=3.9 --pytorch=v2.7.1-7.3.0
      • Adjust according to the target Python / PyTorch version --python and --pytorch; for the version correspondence, see references/reference.md
    • Install:
      • cd dist && pip install dist/torch_npu-*.whl
      • After installation, re-run the pre-checks in Section 1 to confirm that torch_npu is available and npu available: True

Dependencies include:torch_npu, CANN. When building, prefer using the torch_npu official Docker environment.op-plugin For the version matrix of branches with PyTorch/Python/GCC, see references/reference.md。

3. File placement rules: Plan A and Plan B

  • Plan A (preferred, lightweight)
    • Withinthe operator implementation projectcomplete the integration.
    • If the project already has a framework/ directory, prioritize placing the integration files into it.
    • Ifthere is no framework/, then create a separate directory, for example {project_root}/{op_name}_extension/ or {project_root}/framework/{op_name}_extension/。
    • Unless the integration itself genuinely requires modifying the operator implementation, try not to change the existing op_host/、op_kernel/, tiling, json, and other implementation files.
    • Include only the minimal necessary integration artifacts: host wrapper, Python loading/wrapping, build scripts, tests, README.
  • Plan B (op-plugin reference/fallback)
    • Use the op-plugin repository as the integration workspace.
    • Newly generated operator registration or host-side .cpp files should be placed into op_plugin/ops/ in an appropriate subdirectory, and the directory choice should follow the operator domain and existing naming conventions.
    • Other integration process files, such as demo wrappers,setup.py, tests, sample scripts, README, etc., should be placed into example/(or the repository's existing example directory convention).
    • Treat op-plugin as areference implementation path, not as the default location.

4. Integration Mode Selection

General process for adding a new operator: first determinewhether the target operator already exists(for example, built into CANN/ops-nn, or already implemented in xpu_kernel / custom ops) → if it can be reused, prefer Pattern C (OpCommand) → otherwise, depending on whether workspace/tiling is needed, implement your own AscendC kernel in Pattern A or Pattern B.

  • If the operator already has a complete implementation(for example, the CANN built-in operator layer_norm_v3, or an operator in the xpu_kernel / custom ops repository that you have already built and installed):
    • There is no need to compile a new AscendC kernel;
    • you only need to add a lightweight host wrapper in the custom extension, call the graph operator name through at_npu::native::OpCommand , and expose it as torch.ops.npu.*；
    • CMake only needs to link torch_npu; workspace/tiling is handled internally by the underlying graph operator.
    • For the xpu_kernel operator: the graph operator name comes from op_def in OP_ADD(OpClassName). The prerequisite is that xpu_kernel has already been built and installed; otherwise CANN cannot load the graph operator at runtime.
  • If the operator does not yet exist and you only have your own AscendC kernel:
    • Then continue with Pattern A or Pattern B, implementing the kernel, tiling, and host wrapper yourself.

Patterns A, B, and C can coexist in the same project. The core principle is:when existing system capabilities can be reused, prioritize reuse and do not reinvent the wheel.

Pattern A: No workspace (reference add）

  • Kernel： has only inputs and outputs (and optional scalars), and does not require workspace/tiling. Files are generally placed in csrc/kernel/{kernel_name}_custom.cpp, and the Ascend C process is CopyIn -> Compute -> CopyOut。
  • Host： is only responsible for allocating outputs (for example at::empty_like(x) or at::empty(...)), and then calls EXEC_KERNEL_CMD({kernel_name}, blockDim, input..., output[, scalars]). It needs to include aclrtlaunch_{kernel_name}.h。
  • CMake： Use ascendc_library(no_workspace_kernel STATIC csrc/kernel/{kernel_name}_custom.cpp)(or define a separate target for each kernel), and link that library into the final shared extension.

Pattern B: Has workspace and/or tiling (reference matmul_leakyrelu）

  • Kernel： Requires workspace (and optionally tiling). Files are generally csrc/kernel/{kernel_name}_custom.cpp, likewise following CopyIn -> Compute -> CopyOut. When building, add HAVE_WORKSPACE, and if tiling is used also add HAVE_TILING。
  • Host： is responsible for allocating the output and workspace tensors (the size comes from the platform interface or user input), and if needed also allocating the tiling tensor and calling the tiling generation logic. Finally call EXEC_KERNEL_CMD({kernel_name}, blockDim, input..., output, workspace[, tiling]). Need to include aclrtlaunch_{kernel_name}.h。
  • CMake： uses ascendc_library(workspace_kernel STATIC csrc/kernel/{kernel_name}_custom.cpp), and via ascendc_compile_definitions(workspace_kernel PRIVATE -DHAVE_WORKSPACE -DHAVE_TILING) add macro definitions (if tiling is not used, remove HAVE_TILING). If tiling logic exists, also add csrc/host/tiling/*.cpp to the host source file list, and link workspace_kernel into the final shared extension.

Pattern C: Reuse an existing operator (OpCommand pattern)

When the target operator is already fully implemented (for example, built into CANN, or xpu_kernel / custom ops are installed and available), use Pattern C:

  • Basic idea: directly via OpCommand call the graph operator name, without adding a new AscendC kernel.

  • OpCommand input/output naming: If the graph operator is in op_def defines explicit input/output names, they must be matched exactly via the second parameter descName , for example .Input(tensor, "inputGradY")、.Output(tensor, "outputGradX"), so that it can map correctly to the graph operator.

  • Host layer example (LayerNormV3, built into CANN)

    • PyTorch API design:
      • torch.ops.npu.layer_norm_v3(x, gamma, beta, begin_norm_axis, begin_params_axis, eps) -> (y, mean, rstd)
    • Implementation points:
      • Ensure x is on the NPU:x.device().type() == PrivateUse1
      • Construct y/mean/rstd output:
        • y: at::empty_like(x)
        • mean/rstd: shape is [A1...Ai, 1...1], where i = begin_norm_axis
      • use OpCommand：
        • .Name("LayerNormV3")
        • .Input(x).Input(gamma).Input(beta)
        • .Output(y).Output(mean).Output(rstd)
        • .Attr("begin_norm_axis", (int64_t)begin_norm_axis)
        • .Attr("begin_params_axis", (int64_t)begin_params_axis)
        • .Attr("epsilon", (float)eps)
        • .Run()

  • Host layer example (LayerNormV4）

    • PyTorch API design:
      • torch.ops.npu.layer_norm_v4(x, int[] normalized_shape, Tensor? gamma=None, Tensor? beta=None, float eps=1e-5) -> (Tensor, Tensor, Tensor)
    • Implementation points:
      • C++ signature uses at::IntArrayRef normalized_shape、c10::optional<at::Tensor> gamma_opt/beta_opt
      • output:
        • y = at::empty_like(x)
        • mean/rstd has shape [A1...Ai, 1...1], where Ai is the axis not participating in normalization (that is, the dimensions before input.dim() - normalized_shape.size() )
      • OpCommand call method:
        • .Name("LayerNormV4")
        • .Input(x)
        • .Input(normalized_shape) // host-side int list,OpCommand will automatically handle H2D
        • Optional input handling:
          • If gamma_opt has a value:.Input(*gamma_opt), otherwise .Input()(an empty input will map to OPTIONAL_INPUT）
          • If beta_opt has a value:.Input(*beta_opt), otherwise .Input()
        • .Output(y).Output(mean).Output(rstd)
        • .Attr("epsilon", (float)eps)
        • .Run()

  • CMake simplification method:

    • No need for ascendc_library, and no need for the tiling source file; keep only the host source:
      • file(GLOB _SRCS csrc/host/*.cpp)
      • add_library({pkg} SHARED ${_SRCS})
    • Link only the necessary libraries:
      • target_link_libraries({pkg} PRIVATE torch_npu)
    • Add include directories:
      • ${TORCH_NPU_PATH}/include
      • ${TORCH_PATH}/include and torch/csrc/api/include
      • ${ASCEND_CANN_PACKAGE_PATH}/include(for dependencies such as graph/types.h )
      • Add ${ASCEND_CANN_PACKAGE_PATH}/runtime/includeif necessary; in some environments, headers related to OpCommand will continue to include runtime-side graph/* dependencies
    • Practical advice: For host-only Pattern C, if the PyTorch in the current environment does not export TorchConfig.cmake, do not strongly depend on find_package(Torch); at this point, the usually safer approach is to directly use setup.py in torch.utils.cpp_extension.load(...) or an equivalent method to compile the shared library, and then load it through torch.ops.load_library(...) .

  • For xpu_kernel / custom ops, Pattern C description:

    • The graph operator name comes from op_def in OP_ADD(OpClassName)(for example, MoeInitRoutingGroupedMatmulGrad）
    • The input/output names come from definitions such as this->Input("inputGradY")、this->Output("outputGradX") ; when calling, write it as .Input(t, "inputGradY")、.Output(t, "outputGradX")
    • The output shape needs to be derived on the host side according to the infer shape logic (for example, batch = expanded_row_idx.numel() / topk), and ensure it is consistent with the derivation result of the graph operator.
    • The precondition is that xpu_kernel(or custom ops) has already been built and installed, otherwise OpCommand will report at runtime: "op not found"
    • Plan A placement: Prefer putting the host-only wrapper into the operator implementation project's framework/ or a newly created {op_name}_extension/。
    • Plan B placement: If using op-plugin, put the host/registration .cpp into an appropriate subdirectory of op_plugin/ops/ , and put the remaining integration files into example/。
    • Engineering recommendations (from an out-of-container driver case)：
      • setup.py must be completed with torch_npu/include and CANN include (otherwise OpCommand.h / graph/types.h will fail to compile).
      • At the linking stage, libtorch_npu.somust be linked (otherwise at runtime undefined symbol OpCommand::Name）
      • Verification must be performed outside the source directory (to avoid importing the local package causing .so not to be found).

  • Common notes:

    • OpCommand::Attr For the integer parameters of int64_t, it is recommended to uniformly use OpAttrMaker::Set , otherwise bool / int64_t may have overload ambiguity.
    • normalized_shape should be passed from the Python side as List[int], do not pass Tensor; on the C++ side use IntArrayRef
    • For optional tensor parameters, prefer using Tensor? with an empty .Input(), do not use a placeholder tensor instead.
    • If a C API such as aclnn_layer_norm* already exists at the lower layer, prefer calling the graph operator through OpCommand , rather than manually handling aclTensorDesc / aclDataBuffer / aclnn*GetWorkspaceSize
    • setup.py： Use os.F_OK and os.X_OK, do not write a nonexistent os.path.F_OK
    • Do not mistakenly import the TORCH_LIBRARY_* registration library as a Python extension module: If your .so is only responsible for registering torch.ops.npu.*, it usually does not provide a PyInit_* entry point; in this case, treat it as an ordinary shared library and load it through torch.ops.load_library(...) , rather than import _C
    • If using torch.utils.cpp_extension.load(...) to generate the Pattern C shared library, you should usually explicitly use is_python_module=False

Core conclusion:First determine whether the operator already exists (CANN or xpu_kernel); if it already exists, usually you only need to add a PyTorch host wrapper.

5. Kernel implementation

  • Add csrc/kernel/{kernel_name}_custom.cpp. The Ascend C logic follows CopyIn -> Compute -> CopyOut. Pattern A has no workspace; Pattern B uses workspace/tiling according to Ascend C docs .
  • The kernel entry name must be consistent with the name used in host and CMake: the generated header file is aclrtlaunch_{kernel_name}.h, and the host side calls EXEC_KERNEL_CMD({kernel_name}, ...)。
  • Add the corresponding ascendc_libraryin CMake. For Pattern B, additionally add ascendc_compile_definitions via -DHAVE_WORKSPACE, and optionally -DHAVE_TILING. Make sure this library is linked into the final shared library (e.g., op_extension / lib{pkg}.so）。

6. PyTorch Integration: Host Layer

  • Add csrc/host/{op_name}.cpp, which contains:
    • Aten IR definition: TORCH_LIBRARY_FRAGMENT(npu, m) { m.def("{op_name}(...) -> ..."); }
    • Implementation function: For example run_xxx, responsible for allocating the output; if it is Pattern B, you also need to prepare the workspace and tiling, and finally call EXEC_KERNEL_CMD({kernel_name}, blockDim, ...)
    • Registration code: TORCH_LIBRARY_IMPL(npu, PrivateUse1, m) { m.impl("{op_name}", TORCH_FN(run_xxx)); }
  • It is recommended to reuse the utils.hfrom the cpp_extension example (e.g., EXEC_KERNEL_CMD、ConvertTypes and other helper logic). The scalar passed to EXEC_KERNEL_CMD must be anlvalue, do not pass an rvalue directly.
  • For Pattern B, the host implementation must also be responsible for computing the workspace size (e.g., obtained through the platform interface), calling the tiling generation logic, and passing the workspace tensor together with the tiling tensor to EXEC_KERNEL_CMD。

7. Build and Run

  1. SOC_VERSION： In CMakeLists.txt , set set(SOC_VERSION "Ascendxxxyy" ...). The chip name can be obtained via npu-smi info , generally written as Ascend + chip name (e.g., Ascend910B）。
  2. Build the wheel: python setup.py bdist_wheel
  3. Install (force overwrite the old version):
    • cd dist && pip install --force-reinstall *.whl
    • This avoids mistakenly using the old version of {pkg}in site-packages, which would cause the new operator to be "hidden"
  4. Run the test: cd test && python test.py

Notes:

  • In Plan A , these commands are usually executed in the operator implementation project's framework/ or a standalone integration directory
  • In Plan B , these commands are usually executed in the op-plugin example workspace that wraps this operator
  • If the environment is in a container, the actual execution location of the above commands should beinside the container; the host machine is only responsible for preparing files, issuing the docker exec/podman exec commands, and reading artifacts from the shared mounted directory

8. Writing Tests

  • Unified steps: import {pkg}(load .so and complete registration) → construct input tensors (you can first create them on the CPU and then .npu(), or directly create them on the NPU) → call torch.ops.npu.{op_name}(...) → compute the CPU reference value cpu_ref(an existing PyTorch operator or an equivalent formula) → use TestCase.assertRtolEqual(output, cpu_ref)(or an equivalent assertion within the project) for comparison.
  • Pattern A style (e.g., add）： cpu_ref = torch.add(x, y)(or the corresponding equivalent PyTorch implementation)
  • Pattern B style (e.g., matmul_leakyrelu）： cpu_ref = some_combination(e.g. LeakyReLU(matmul(a,b) + bias))
  • For each new operator, add a new test method in test/test.py (e.g., test_xxx), and just follow the above pattern. There is no need to write an additional separate demo.py; the test uniformly serves as the entry point for running and verification.
  • If it depends on torch_npu.testing.testcase.TestCase, please install expecttest：pip install expecttest
  • When doing import or registration checks, try not to run tests directly in thesource package root directory; otherwise, the local source directory may take precedence over site-packages and be imported by Python first, causing a false determination of "import failure" or a circular import. A more reliable approach is:
    • First install the wheel
    • Then execute from outside the project directory python -c 'import {pkg}' or run the test

9. Required files and directory layout

Placeholder description:{pkg} represents the Python package name,{kernel_name} represents the kernel entry name,{op_name} represents the PyTorch API name.Naming must be consistent throughout. For example, if the kernel is called add_custom, then {op_name} should usually also be written as add_custom。

Plan A / Pattern A/B(with kernel, recommended layout):

<project_root>/
├── framework/                      # if the project already has one, use it first
│   └── {pkg}/
│       ├── {pkg}/
│       │   ├── __init__.py
│       │   └── _load.py
│       ├── csrc/
│       │   ├── kernel/
│       │   │   └── {kernel_name}_custom.cpp
│       │   └── host/
│       │       ├── {op_name}.cpp
│       │       ├── utils.h
│       │       └── tiling/        # 可选，Pattern B 使用
│       │           └── *_tiling.cpp
│       ├── CMakeLists.txt
│       ├── setup.py
│       └── test/
│           └── test.py
└── ...                             # 原有 op_host/op_kernel/json 保持原位

If there is no framework/, create a new {project_root}/{op_name}_extension/in the same directory, keeping the internal structure consistent.

Plan A / Pattern C(host only, no kernel):

<project_root>/
├── framework/ or {op_name}_extension/
│   ├── {pkg}/
│   │   ├── __init__.py
│   │   └── _load.py
│   ├── csrc/host/
│   │   └── {op_name}.cpp     # OpCommand only
│   ├── CMakeLists.txt
│   ├── setup.py
│   └── test/
│       └── test.py
└── ...                        # 算子实现继续保留在原有源码树中

Plan B(based on the op-plugin placement layout):

<op-plugin-root>/
├── op_plugin/
│   └── ops/
│       └── <chosen_domain_folder>/
│           └── {op_name}.cpp      # host/registration .cpp，按领域归类放置
├── example/ or examples/
│   └── {op_name}_extension/
│       ├── {pkg}/
│       ├── csrc/
│       ├── CMakeLists.txt
│       ├── setup.py
│       ├── test/
│       └── README.md
└── ...

Naming consistency requirements: {kernel_name} must match both aclrtlaunch_{kernel_name}.h、EXEC_KERNEL_CMD({kernel_name}, ...) and the kernel source file name. The package name must also be consistent with the generated .so name and the name in setup.py .

For multiple operators: Under Pattern A/B, usually each kernel corresponds to one ascendc_library, each operator corresponds to one csrc/host/{op_name}.cpp, and a corresponding test.py is added in test_xxx。

10. End-to-end checklist

If you want to adopt acomplete, automated, close to demo-styleprocess, you can execute in the following order:

  1. First determine the placement plan:
    • If you already have an operator implementation project, prefer using Plan A
    • Only switch to op-plugin delivery, or when Plan A is not suitable Plan B
  2. Prepare the environment and torch_npu(usually done only once per machine):
    • source <CANN install path>/set_env.sh
    • Execute the pre-checks in Section 1
    • If torch_npu is missing or unavailable, and Plan B is currently adopted, build and install according to Section 2
  3. Project build (execute after each modification of kernel / host / CMake):
    • In CMakeLists.txt set a supported SOC_VERSION(for example, refer to the host_config.cmake in the CANN support list ascend910b2）
    • python setup.py bdist_wheel
    • cd dist && pip install --force-reinstall *.whl
  4. Run tests (execute after each operator modification):
    • In test/test.py , as described in Section 8:
      • import {pkg}(automatically load lib{pkg}.so）
      • Call torch.ops.npu.{op_name}(...), where op_name must match the actual operator name, do not keep it as my_*
      • Compute the CPU reference value cpu_ref
      • Use assertRtolEqual to compare results
    • Execute:cd test && python test.py
  5. Quickly confirm whether it is registered (optional debugging step):
    • python - << 'EOF'
    • import torch, {pkg} # noqa
    • print([name for name in dir(torch.ops.npu) if "{op_name_hint}" in name])
    • EOF
    • When the test reports "no attribute", you can use this method to quickly confirm whether the operator has actually been registered successfully.

11. References

The following documents provide supplementary information; the actual main process is still based on the current SKILL.md .

12. Supplementary resources

  • references/reference.md — Version table,SOC_VERSION、op-plugin and cpp_extension README links, Ascend C materials
  • references/examples.md — General checklist for adding operators (select mode → kernel/host → CMake → test)