SuperAPI AI Directory

atb-aclnn-operator-migration

Coding & development

View repository

Updated 10-24 04:38

About this skill

ATB to ACLNN Operator Migration Tool

Function Overview

This tool helps developers migrate operators in ATB to ACLNN implementations, especially to achieve better compatibility optimization on 910B/950 devices.

When to Invoke

Invoke this skill in the following situations:

  • The user needs to migrate ATB operators to ACLNN implementations to improve performance on 910B/950 devices
  • The user has generated an operator replacement design document (it is recommended to first run atb-aclnn-operator-replacement-designer)
  • The user needs to apply ACLNN Runner modifications to the ATB source code

⚠️ Path Constraints (Mandatory)

Before executing this skill,the following paths must be obtained from the user:

  • <CANN_PATH>: CANN installation path (e.g., /usr/local/Ascend/ascend-toolkit/latest)
  • <ATB_REPO_PATH>: ATB repository path (e.g., {your working path}ascend-transformer-boost)

If the user does not provide them or the paths are invalid, stop immediately and report an error.


Prerequisites

  • CANN environment is installed (ASCEND_TOOLKIT_HOME is set)
  • The ATB source repository has been cloned locally
  • The ATB interface documentation and ACLNN interface documentation corresponding to the operator have been obtained
  • It is recommended to have completed atb-aclnn-operator-replacement-designer to generate the design document

At the end of each operation step, the checkpoint marks the verification criteria. After execution is complete, please fill in the actual status in the Execution Results section.

Checkpoint ID Description Verification Command
Step0-ENV CANN environment sourced echo $ASCEND_TOOLKIT_HOME
Step0-DIR ATB source directory exists [ -d <ATB_REPO_PATH> ]
Step1-DIR Original Operation source file located ls <op>_operation.cpp
Step1-DIR Existing ACLNN runner reference located ls softmax_aclnn_runner.cpp
Step2-FILE <op>_aclnn_runner.h created [ -f <op>_aclnn_runner.h ]
Step2-FILE <op>_aclnn_runner.cpp created [ -f <op>_aclnn_runner.cpp ]
Step2-CODE BuildAclnnVariantPack implemented grep "BuildAclnnVariantPack" <op>_aclnn_runner.cpp
Step2-CODE SetAclNNWorkspaceExecutor implemented grep "SetAclNNWorkspaceExecutor" <op>_aclnn_runner.cpp
Step2-CODE LaunchAclnnKernel implemented grep "LaunchAclnnKernel" <op>_aclnn_runner.cpp
Step2-CODE LoadMethod implemented grep "LoadMethod" <op>_aclnn_runner.cpp
Step3-MOD CreateRunner contains 910B/950 branches grep "Is910B|Is950|910B|950" <op>_operation.cpp
Step4-BUILD Compilation succeeds with no errors make ... 2>&1
Step5-TEST Functions normally on non-910B/950 devices ./apitest ...

Migration Steps

Step 1: Analyze the ATB operator implementation

  1. Understand the ATB operator parameter structure

    • View the parameter structure in infer_op_params.h parameter structure in
    • Confirm parameter types and constraints
  2. Analyze the existing implementation

    • View <operator>_operation.cpp to understand the operation logic
    • Analyze <operator>_ops_runner.cpp to understand the existing runner implementation
  3. Study the ACLNN interface

    • Understand the interface of the corresponding ACLNN operator
    • Analyze the parameter mapping relationships

Step 2: Create the ACLNN Runner

  1. Create the header file (<operator>_aclnn_runner.h)

    • Inherit from the AclnnRunner base class
    • Define the necessary member variables and methods
    • Declare the ACLNN function pointers
  2. Implement the source file (<operator>_aclnn_runner.cpp)

    • Implement the BuildAclnnVariantPack method to handle tensor conversion
    • Implement the SetAclNNWorkspaceExecutor method to configure the workspace
    • Implement the LaunchAclnnKernel method to execute the computation
    • Implement the LoadMethod method to dynamically load the ACLNN functions

Step 3: Modify the Operation implementation

  1. Update header file includes

    • Add #include "<operator>_aclnn_runner.h"
  2. Modify the CreateRunner method

    • Add device detection logic
    • Use on 910B/950 devices <Operator>AclnnRunner
    • Keep the original on other devices <Operator>OpsRunner
  3. Update device support

    • Modify the CreateOperation function
    • Support 910B/950 training devices
    • Maintain support for other devices

Step 4: Build and verify

  1. Compilation verification

    • Ensure the code compiles successfully
    • Check for compilation errors or warnings
  2. Functional verification

    • Verify the ACLNN implementation on 910B/950 devices
    • Verify the original implementation on other devices
    • Ensure functional correctness
  3. Performance verification

    • Compare the performance of the ATB and ACLNN implementations
    • Ensure performance gains are achieved on 910B/950 devices

Core implementation

  1. Parameter mapping

    • Mapping of ATB parameters to ACLNN parameters
    • Handle conversion of input and output tensors
  2. Device detection

    • Use GetSingleton<Config>().Is910B() to detect the 910B device type
    • Use GetSingleton<Config>().Is950() to detect the 950 device type
    • Select the appropriate runner based on the device type
  3. Memory management

    • Automatically manage the ACLNN workspace
    • Handle tensor data pointers properly
  4. Error handling

    • Comprehensive logging
    • Error code handling and return

Performance optimization

  1. ACLNN acceleration

    • Leverage ACLNN hardware optimization
    • Reduce memory access overhead
  2. Conditional compilation

    • Select the optimal implementation based on the device type
    • Maintain code compatibility

Notes

  1. Device compatibility

    • Ensure correct operation on different devices
    • Handle device-specific parameters and constraints
  2. Interface consistency

    • Keep the original ATB interface unchanged
    • Ensure upper-layer applications require no modification
  3. Test coverage

    • Write test cases for different devices
    • Verify performance and functional correctness
  4. Error handling

    • Comprehensive error handling mechanism
    • Detailed logging

Example usage

Step 1: Prepare the environment

# 确保CANN环境正确配置
# 确保ATB代码库已更新

Step 2: Apply the migration

# 在ATB代码库中应用迁移
git apply <operator>_aclnn_migration.patch

Step 3: Compilation verification

# 编译ATB代码
cd ascend-transformer-boost
mkdir build && cd build
cmake ..
make -j

Step 4: Functional testing

# 运行算子测试
./tests/apitest/opstest/python/operations/<operator>/test_<operator>.py

Step 5: Performance testing

# 对比ATB和ACLNN实现的性能
python performance_benchmark.py

Q: What should I do if compilation fails?

A: Check whether the CANN version is compatible and ensure the ACLNN interface is available.

Q: No performance improvement on 910B/950 devices?

A: Check whether the ACLNN functions are loaded correctly and verify the workspace configuration.

Q: Functional issues on other devices?

A: Ensure the device detection logic is correct and verify that the original implementation is unaffected.

Summary

With this tool, developers can easily migrate ATB operators to ACLNN implementations, achieving better performance on 910B/950 devices while maintaining compatibility with other devices. The implementation follows the project code specifications and remains consistent with existing ACLNN runner implementations, ensuring code quality and maintainability.

Execution Results

⚠️ Fill in after execution: After the skill execution is complete, fill in the actual execution results according to the format below.

Checkpoint Checklist

Step Checkpoint Description Status
1 ATB source code directory exists ✅ PASS / ❌ FAIL / ⏭️ SKIP
1 Original Operation source file located ✅ PASS / ❌ FAIL / ⏭️ SKIP
1 Existing ACLNN runner reference located ✅ PASS / ❌ FAIL / ⏭️ SKIP
2 <op>_aclnn_runner.h Create ✅ PASS / ❌ FAIL / ⏭️ SKIP
2 <op>_aclnn_runner.cpp Create ✅ PASS / ❌ FAIL / ⏭️ SKIP
2 BuildAclnnVariantPack Implement ✅ PASS / ❌ FAIL / ⏭️ SKIP
2 SetAclNNWorkspaceExecutor Implement ✅ PASS / ❌ FAIL / ⏭️ SKIP
2 LaunchAclnnKernel Implement ✅ PASS / ❌ FAIL / ⏭️ SKIP
2 LoadMethod Implement ✅ PASS / ❌ FAIL / ⏭️ SKIP
3 CreateRunner includes 910B/950 branches ✅ PASS / ❌ FAIL / ⏭️ SKIP
4 Compilation succeeded ✅ PASS / ❌ FAIL / ⏭️ SKIP
5 Functionality on non-910B/950 devices is normal ✅ PASS / ❌ FAIL / ⏭️ SKIP
5 ACLNN path on 910B devices is normal ✅ PASS / ❌ FAIL / ⏭️ SKIP
5 ACLNN path on 950 devices is normal ✅ PASS / ❌ FAIL / ⏭️ SKIP

VERDICT: ✅ SUCCESS / ⚠️ PARTIAL / ❌ FAILED / ⏭️ SKIPPED

Issue List (if any)

Severity Checkpoint Issue Description Suggestion
🔴 CRITICAL - - -
🟡 WARNING - - -

Execution Summary

  • Execution Time:
  • Execution Environment:
  • Operator Name:
  • Duration:
  • Pass Rate: