Function Overview
This tool helps developers migrate operators in ATB to ACLNN implementations, especially to achieve better compatibility optimization on 910B/950 devices.
When to Invoke
Invoke this skill in the following situations:
- The user needs to migrate ATB operators to ACLNN implementations to improve performance on 910B/950 devices
- The user has generated an operator replacement design document (it is recommended to first run
atb-aclnn-operator-replacement-designer) - The user needs to apply ACLNN Runner modifications to the ATB source code
⚠️ Path Constraints (Mandatory)
Before executing this skill,the following paths must be obtained from the user:
<CANN_PATH>: CANN installation path (e.g.,/usr/local/Ascend/ascend-toolkit/latest)<ATB_REPO_PATH>: ATB repository path (e.g.,{your working path}ascend-transformer-boost)
If the user does not provide them or the paths are invalid, stop immediately and report an error.
Prerequisites
- CANN environment is installed (
ASCEND_TOOLKIT_HOMEis set) - The ATB source repository has been cloned locally
- The ATB interface documentation and ACLNN interface documentation corresponding to the operator have been obtained
- It is recommended to have completed
atb-aclnn-operator-replacement-designerto generate the design document
At the end of each operation step, the checkpoint marks the verification criteria. After execution is complete, please fill in the actual status in the Execution Results section.
| Checkpoint ID | Description | Verification Command |
|---|---|---|
| Step0-ENV | CANN environment sourced | echo $ASCEND_TOOLKIT_HOME |
| Step0-DIR | ATB source directory exists | [ -d <ATB_REPO_PATH> ] |
| Step1-DIR | Original Operation source file located | ls <op>_operation.cpp |
| Step1-DIR | Existing ACLNN runner reference located | ls softmax_aclnn_runner.cpp |
| Step2-FILE | <op>_aclnn_runner.h created |
[ -f <op>_aclnn_runner.h ] |
| Step2-FILE | <op>_aclnn_runner.cpp created |
[ -f <op>_aclnn_runner.cpp ] |
| Step2-CODE | BuildAclnnVariantPack implemented |
grep "BuildAclnnVariantPack" <op>_aclnn_runner.cpp |
| Step2-CODE | SetAclNNWorkspaceExecutor implemented |
grep "SetAclNNWorkspaceExecutor" <op>_aclnn_runner.cpp |
| Step2-CODE | LaunchAclnnKernel implemented |
grep "LaunchAclnnKernel" <op>_aclnn_runner.cpp |
| Step2-CODE | LoadMethod implemented |
grep "LoadMethod" <op>_aclnn_runner.cpp |
| Step3-MOD | CreateRunner contains 910B/950 branches | grep "Is910B|Is950|910B|950" <op>_operation.cpp |
| Step4-BUILD | Compilation succeeds with no errors | make ... 2>&1 |
| Step5-TEST | Functions normally on non-910B/950 devices | ./apitest ... |
Migration Steps
Step 1: Analyze the ATB operator implementation
-
Understand the ATB operator parameter structure
- View the parameter structure in
infer_op_params.hparameter structure in - Confirm parameter types and constraints
- View the parameter structure in
-
Analyze the existing implementation
- View
<operator>_operation.cppto understand the operation logic - Analyze
<operator>_ops_runner.cppto understand the existing runner implementation
- View
-
Study the ACLNN interface
- Understand the interface of the corresponding ACLNN operator
- Analyze the parameter mapping relationships
Step 2: Create the ACLNN Runner
-
Create the header file (
<operator>_aclnn_runner.h)- Inherit from the
AclnnRunnerbase class - Define the necessary member variables and methods
- Declare the ACLNN function pointers
- Inherit from the
-
Implement the source file (
<operator>_aclnn_runner.cpp)- Implement the
BuildAclnnVariantPackmethod to handle tensor conversion - Implement the
SetAclNNWorkspaceExecutormethod to configure the workspace - Implement the
LaunchAclnnKernelmethod to execute the computation - Implement the
LoadMethodmethod to dynamically load the ACLNN functions
- Implement the
Step 3: Modify the Operation implementation
-
Update header file includes
- Add
#include "<operator>_aclnn_runner.h"
- Add
-
Modify the CreateRunner method
- Add device detection logic
- Use on 910B/950 devices
<Operator>AclnnRunner - Keep the original on other devices
<Operator>OpsRunner
-
Update device support
- Modify the
CreateOperationfunction - Support 910B/950 training devices
- Maintain support for other devices
- Modify the
Step 4: Build and verify
-
Compilation verification
- Ensure the code compiles successfully
- Check for compilation errors or warnings
-
Functional verification
- Verify the ACLNN implementation on 910B/950 devices
- Verify the original implementation on other devices
- Ensure functional correctness
-
Performance verification
- Compare the performance of the ATB and ACLNN implementations
- Ensure performance gains are achieved on 910B/950 devices
Core implementation
-
Parameter mapping
- Mapping of ATB parameters to ACLNN parameters
- Handle conversion of input and output tensors
-
Device detection
- Use
GetSingleton<Config>().Is910B()to detect the 910B device type - Use
GetSingleton<Config>().Is950()to detect the 950 device type - Select the appropriate runner based on the device type
- Use
-
Memory management
- Automatically manage the ACLNN workspace
- Handle tensor data pointers properly
-
Error handling
- Comprehensive logging
- Error code handling and return
Performance optimization
-
ACLNN acceleration
- Leverage ACLNN hardware optimization
- Reduce memory access overhead
-
Conditional compilation
- Select the optimal implementation based on the device type
- Maintain code compatibility
Notes
-
Device compatibility
- Ensure correct operation on different devices
- Handle device-specific parameters and constraints
-
Interface consistency
- Keep the original ATB interface unchanged
- Ensure upper-layer applications require no modification
-
Test coverage
- Write test cases for different devices
- Verify performance and functional correctness
-
Error handling
- Comprehensive error handling mechanism
- Detailed logging
Example usage
Step 1: Prepare the environment
# 确保CANN环境正确配置
# 确保ATB代码库已更新
Step 2: Apply the migration
# 在ATB代码库中应用迁移
git apply <operator>_aclnn_migration.patch
Step 3: Compilation verification
# 编译ATB代码
cd ascend-transformer-boost
mkdir build && cd build
cmake ..
make -j
Step 4: Functional testing
# 运行算子测试
./tests/apitest/opstest/python/operations/<operator>/test_<operator>.py
Step 5: Performance testing
# 对比ATB和ACLNN实现的性能
python performance_benchmark.py
Q: What should I do if compilation fails?
A: Check whether the CANN version is compatible and ensure the ACLNN interface is available.
Q: No performance improvement on 910B/950 devices?
A: Check whether the ACLNN functions are loaded correctly and verify the workspace configuration.
Q: Functional issues on other devices?
A: Ensure the device detection logic is correct and verify that the original implementation is unaffected.
Summary
With this tool, developers can easily migrate ATB operators to ACLNN implementations, achieving better performance on 910B/950 devices while maintaining compatibility with other devices. The implementation follows the project code specifications and remains consistent with existing ACLNN runner implementations, ensuring code quality and maintainability.
Execution Results
⚠️ Fill in after execution: After the skill execution is complete, fill in the actual execution results according to the format below.
Checkpoint Checklist
| Step | Checkpoint Description | Status |
|---|---|---|
| 1 | ATB source code directory exists | ✅ PASS / ❌ FAIL / ⏭️ SKIP |
| 1 | Original Operation source file located | ✅ PASS / ❌ FAIL / ⏭️ SKIP |
| 1 | Existing ACLNN runner reference located | ✅ PASS / ❌ FAIL / ⏭️ SKIP |
| 2 | <op>_aclnn_runner.h Create |
✅ PASS / ❌ FAIL / ⏭️ SKIP |
| 2 | <op>_aclnn_runner.cpp Create |
✅ PASS / ❌ FAIL / ⏭️ SKIP |
| 2 | BuildAclnnVariantPack Implement |
✅ PASS / ❌ FAIL / ⏭️ SKIP |
| 2 | SetAclNNWorkspaceExecutor Implement |
✅ PASS / ❌ FAIL / ⏭️ SKIP |
| 2 | LaunchAclnnKernel Implement |
✅ PASS / ❌ FAIL / ⏭️ SKIP |
| 2 | LoadMethod Implement |
✅ PASS / ❌ FAIL / ⏭️ SKIP |
| 3 | CreateRunner includes 910B/950 branches | ✅ PASS / ❌ FAIL / ⏭️ SKIP |
| 4 | Compilation succeeded | ✅ PASS / ❌ FAIL / ⏭️ SKIP |
| 5 | Functionality on non-910B/950 devices is normal | ✅ PASS / ❌ FAIL / ⏭️ SKIP |
| 5 | ACLNN path on 910B devices is normal | ✅ PASS / ❌ FAIL / ⏭️ SKIP |
| 5 | ACLNN path on 950 devices is normal | ✅ PASS / ❌ FAIL / ⏭️ SKIP |
VERDICT: ✅ SUCCESS / ⚠️ PARTIAL / ❌ FAILED / ⏭️ SKIPPED
Issue List (if any)
| Severity | Checkpoint | Issue Description | Suggestion |
|---|---|---|---|
| 🔴 CRITICAL | - | - | - |
| 🟡 WARNING | - | - | - |
Execution Summary
- Execution Time:
- Execution Environment:
- Operator Name:
- Duration:
- Pass Rate: