atb-testframework-build

ATB Test Framework Compilation

Feature Overview

This skill provides compilation guidance for the ATB (Ascend Transformer Boost) test framework, including both full compilation (with third-party dependency cloning and source replacement) and incremental compilation modes.

When to Invoke

Invoke this skill in the following situations:

• The user needs to compile the ATB test framework
• The user needs to run CSV tests but the test framework has not been compiled
• The user needs to build the ATB project from scratch
• The user encounters third-party dependency cloning failures

Prerequisites

• The Docker container has been started
• The CANN environment has been installed
• The ATB source repository has been cloned

Key Path Variables

Before compiling, confirm the following path variables (obtained from the user):

Variable	Description	Example
<CANN_PATH>	CANN installation root directory, containing set_env.sh	/path/to/cann
<ATB_REPO_PATH>	ATB source repository root directory	/path/to/ascend-transformer-boost
<DOCKER_NAME>	Docker container name	<YOUR_DOCKER_NAME>

Compilation Mode Selection

Incremental Compilation

When the 3rdparty/ directory already exists and third-party dependencies are complete, directly perform incremental compilation:

docker exec <DOCKER_NAME> bash -c "
source <CANN_PATH>/set_env.sh && \
cd <ATB_REPO_PATH> && \
bash scripts/build.sh testframework
"

Applicable scenario: third-party dependencies are ready, and only the ATB source code needs to be recompiled.

Full Compilation

When the 3rdparty/ directory is empty or does not exist, full compilation is required, including cloning of third-party dependencies. Since GitHub may not be accessible inside the Docker container, source replacement must be performed first.

Step 1: Replace GitHub sources with gitcode mirrors

ATB's scripts/build.sh contains multiple GitHub dependencies, which need to be replaced with gitcode.com mirrors in the domestic network environment.

Source mapping table to be replaced：

Original GitHub URL	gitcode replacement URL	Function in build.sh
https://github.com/nlohmann/json.git	https://gitcode.com/nlohmann/json.git	fn_build_nlohmann_json
https://github.com/google/googletest.git	https://gitcode.com/google/googletest.git	fn_build_googletest
https://github.com/coolxv/cpp-stub.git	https://gitcode.com/coolxv/cpp-stub.git	fn_build_stub
https://github.com/pybind/pybind11.git	https://gitcode.com/pybind/pybind11.git	fn_build_pybind11
https://github.com/doxygen/doxygen/releases/download/Release_1_9_6/doxygen-1.9.6.src.tar.gz	Use git clone --branch Release_1_9_6 --depth 1 https://gitcode.com/doxygen/doxygen.git to replace wget	fn_build_doxygen

Replacement commands：

cd <ATB_REPO_PATH>
sed -i 's|https://github.com/nlohmann/json.git|https://gitcode.com/nlohmann/json.git|g' scripts/build.sh
sed -i 's|https://github.com/google/googletest.git|https://gitcode.com/google/googletest.git|g' scripts/build.sh
sed -i 's|https://github.com/coolxv/cpp-stub.git|https://gitcode.com/coolxv/cpp-stub.git|g' scripts/build.sh
sed -i 's|https://github.com/pybind/pybind11.git|https://gitcode.com/pybind/pybind11.git|g' scripts/build.sh

doxygen replacement notes: doxygen's wget download needs to be changed to git clone; modify the fn_build_doxygen function:

# 将 wget + tar 方式替换为 git clone
# 原始代码:
#   [[ ! -f "doxygen-1.9.6.src.tar.gz" ]] && wget --no-check-certificate https://github.com/doxygen/doxygen/releases/download/Release_1_9_6/doxygen-1.9.6.src.tar.gz
#   tar -xzvf doxygen-1.9.6.src.tar.gz
#   mv doxygen-1.9.6 doxygen
# 替换为:
#   [[ ! -d "doxygen" ]] && git clone --branch Release_1_9_6 --depth 1 https://gitcode.com/doxygen/doxygen.git

Step 2: Clean old build artifacts (optional)

cd <ATB_REPO_PATH>
rm -rf build output 3rdparty

Step 3: Perform full compilation

docker exec <DOCKER_NAME> bash -c "
source <CANN_PATH>/set_env.sh && \
cd <ATB_REPO_PATH> && \
bash scripts/build.sh testframework
"

Compilation Artifact Verification

After successful compilation, check the key artifacts:

# 检查测试框架库
ls -lh <ATB_REPO_PATH>/output/atb/cxx_abi_0/lib/libatb_test_framework.so

# 检查 ATB 主库
ls -lh <ATB_REPO_PATH>/output/atb/cxx_abi_0/lib/libatb.so

Expected artifacts：

• libatb_test_framework.so — test framework core library
• libatb.so — ATB main library
• atb_testframework_<arch>_<timestamp>.tar.gz — package file

Post-compilation Environment Setup

After compilation is complete, set the environment variables to run CSV tests:

source <ATB_REPO_PATH>/output/atb/set_env.sh

Or set manually:

export ATB_HOME_PATH=<ATB_REPO_PATH>/output/atb/cxx_abi_0
export LD_LIBRARY_PATH=<ATB_REPO_PATH>/output/atb/cxx_abi_0/lib:$LD_LIBRARY_PATH

Running CSV Tests

cd <ATB_REPO_PATH>/tests/framework/python/CsvOpsTestTool
python3 atb_csv_ops_test.py -i <csv_file_path> -n <start>:<end>

testframework third-party dependencies required for the build:

Dependency	Source	Purpose
nlohmannJson	gitcode.com/nlohmann/json (v3.11.3)	JSON parsing library
Mind-KernelInfra (mki)	gitcode.com/cann/ascend-boost-comm	Kernel infrastructure
catlass	gitcode.com/cann/catlass	Matrix computation library
compiler	CANN path symbolic link	Compiler toolchain

Note：testframework googletest, cpp-stub, pybind11, and doxygen are not required (these are needed for unittest/fuzztest/doc build modes). However, if other build modes are used (such as alltest、unittest), the corresponding GitHub sources also need to be replaced.

CXX ABI version

At compile time, USE_CXX11_ABI is automatically detected by the fn_init_env function:

• Check whether the system-installed torch uses the CXX11 ABI
• The output directory is output/atb/cxx_abi_0/ or output/atb/cxx_abi_1/

Problem	Cause	Solution
git clone failed (curl 16)	GitHub network unreachable	Replace with the gitcode mirror source
ASCEND_HOME_PATH is null	CANN environment not sourced	First run source <CANN_PATH>/set_env.sh
libatb_test_framework.so does not exist	Compilation not completed or failed	Check the build log to ensure BUILD_TEST_FRAMEWORK=ON
Third-party dependency clone 403	Incorrect gitcode repository path	Verify the repository path and tag on gitcode

Recompile after modifying files

After modifying *_aclnn_runner.cpp、*_operation.cpp and other source code, recompilation is required:

# 在 Docker 容器内执行（已在容器中）
source $ASCEND_TOOLKIT_HOME/set_env.sh
cd $ATB_REPO_PATH
bash scripts/build.sh testframework 2>&1 | tail -50

Note: Ensure the environment variables are set:

• ASCEND_TOOLKIT_HOME(CANN installation path, such as /usr/local/Ascend/ascend-toolkit/latest）
• ATB_REPO_PATH(ATB repository path, such as {your working path}ascend-transformer-boost）

Compilation failure handling process：

Failure count	Action	Description
1st failure	Analyze the log, locate the problem, and recompile after fixing	Forward analysis of the error cause
2nd failure	rm -rf build/ output/ clean and recompile	Clean the cmake cache
3rd failure	bash scripts/build.sh testframework --clean-first	Compile from scratch with a full clean

Compilation constraint rules：

• Verify at most 3 times: If all 3 attempts fail, record the complete error information and ask the user
• Step by Step: Fix only one problem at a time, then recompile and verify
• When unable to resolve: Record the problem and ask the user

Compilation error record template：

### 编译失败记录 (#N)

**错误信息**：

<compiler output>

**问题分析**：
<分析原因>

**修改计划**：
1. <修改步骤1>
2. <修改步骤2>

**状态**：待用户确认

Execution results

⚠️ Fill in after execution: After the skill execution is complete, fill in the actual execution results according to the format below.

Checkpoint checklist

Step	Checkpoint description	Status
-	CANN environment sourced (ASCEND_TOOLKIT_HOME is set)	✅ PASS / ❌ FAIL / ⏭️ SKIP
-	Docker container running	✅ PASS / ❌ FAIL / ⏭️ SKIP
-	ATB source directory exists	✅ PASS / ❌ FAIL / ⏭️ SKIP
1	GitHub-to-gitcode source replacement applied	✅ PASS / ❌ FAIL / ⏭️ SKIP
2	libatb.so build artifact exists	✅ PASS / ❌ FAIL / ⏭️ SKIP
2	libatb_test_framework.so build artifact exists	✅ PASS / ❌ FAIL / ⏭️ SKIP
2	CXX ABI directory naming is correct	✅ PASS / ❌ FAIL / ⏭️ SKIP
3	ATB_HOME_PATH points to the output directory	✅ PASS / ❌ FAIL / ⏭️ SKIP
3	LD_LIBRARY_PATH contains the ATB lib	✅ PASS / ❌ FAIL / ⏭️ SKIP
4	CSV tests can start normally	✅ PASS / ❌ FAIL / ⏭️ SKIP

VERDICT: ✅ SUCCESS / ⚠️ PARTIAL / ❌ FAILED / ⏭️ SKIPPED

Problem list (if any)

Level	Checkpoint	Problem description	Suggestion
🔴 CRITICAL	-	-	-
🟡 WARNING	-	-	-

Execution summary

• Execution time：
• Execution environment：
• ATB_REPO_PATH：
• Duration：
• Pass rate：