ai-for-science-ascend-tf-community

Ascend TensorFlow Community Migration and Adaptation Skill

Project Overview

This Skill documents the complete process of deploying TF 2.x models on Ascend NPU using the TensorFlow Community approach (not TF→PyTorch conversion). The core idea is: keep the original TF code essentially unchanged, and through the npu_device plugin, dispatch operators to execute on the Ascend NPU.

Applicable scenarios:

• Existing TF 2.x training/inference code that you want to run directly on Ascend NPU
• Do not want or it is not suitable to perform TF→PyTorch framework conversion
• Reference documents:https://www.hiascend.com/document/detail/zh/TensorFlowCommunity/850/migration/tfmigr2/

---

Prerequisites

Item	Requirement
Hardware	Ascend910 series (at least 1 card)
CANN	≥ 8.0 (8.5 recommended)
Python	3.7.x / 3.8.x / 3.9.x
TensorFlow	2.6.5 (requires source compilation, ABI=0)
npu_device	0.1 (from Ascend-cann-tfplugin)
Bazel	3.7.2 (required for compiling TF)

---

Migration Process Overview

1. 环境检查（NPU、CANN、GCC）
→ 2. 源码编译 TF 2.6.5（aarch64 专用流程）
→ 3. 安装 tfplugin（npu_device）
→ 4. 代码适配（自动迁移工具 or 手动）
→ 5. 推理/训练验证
→ 6. 精度对比

---

Step 1: Environment Check

# 检查 NPU 设备
npu-smi info

# 检查 CANN 版本
cat /usr/local/Ascend/ascend-toolkit/latest/version.cfg 2>/dev/null || \
cat /home/Ascend/ascend-toolkit/latest/version.cfg 2>/dev/null

# CANN 环境变量（根据实际安装路径选择）
source /home/Ascend/8.5/cann-8.5.0/set_env.sh
# 或
# source /usr/local/Ascend/ascend-toolkit/set_env.sh

# 检查编译器
gcc --version   # 需要 GCC 7.3+
uname -m        # aarch64 需要源码编译 TF

---

Step 2: Source Compilation of TF 2.6.5 (aarch64)

Key: The aarch64 platform does not provide a TF 2.6.x wheel via pip, so it must be compiled from source. The x86_64 platform can directly pip install tensorflow-cpu==2.6.5。

2.1 Create Conda Environment

conda create -n tf_npu python=3.8 -y
conda activate tf_npu
conda install -y hdf5
pip install numpy==1.21.6 keras-preprocessing packaging h5py==3.7.0

2.2 Install Bazel 3.7.2

# 从华为镜像下载（GitHub 可能超时）
wget 'https://mirrors.huaweicloud.com/bazel/3.7.2/bazel-3.7.2-linux-arm64' \
     -O /usr/local/bin/bazel
chmod +x /usr/local/bin/bazel
bazel version  # 确认 Build label: 3.7.2

2.3 Clone TF Source + nsync Patch

git clone --depth 1 --branch v2.6.5 \
    https://github.com/tensorflow/tensorflow.git tf265_src
cd tf265_src

nsync aarch64 patch(required by Ascend documentation):

# 下载 nsync 源码
wget 'https://github.com/google/nsync/archive/1.22.0.tar.gz' \
     -O /tmp/nsync-1.22.0.tar.gz
cd /tmp && tar -zxf nsync-1.22.0.tar.gz

Edit /tmp/nsync-1.22.0/platform/c++11/atomic.h, at NSYNC_CPP_START_ add after:

#define ATM_CB_() __sync_synchronize()

And in each atm_cas_*_u32_ function, change return (std::atomic_compare_exchange_strong_explicit(...)); to:

int result = (std::atomic_compare_exchange_strong_explicit(...));
ATM_CB_();
return result;

Repackage and update sha256:

cd /tmp && rm -f nsync-1.22.0.tar.gz
tar -czf nsync-1.22.0.tar.gz nsync-1.22.0/
sha256sum nsync-1.22.0.tar.gz   # 记录新的 hash

Modify the nsync tensorflow/workspace2.bzl in sha256 and urls(add file:///tmp/nsync-1.22.0.tar.gz）。

2.4 Configure Compilation

# 设置环境变量
export PYTHON_BIN_PATH=$(which python)
export TF_NEED_CUDA=0
export TF_NEED_ROCM=0
export TF_ENABLE_XLA=0
export CC_OPT_FLAGS='-march=armv8-a'

yes '' | ./configure

Key: set ABI=0(Ascend documentation requires TF and FwkPlugin ABI consistency):

At the top of .tf_configure.bazelrc add:

build --cxxopt="-D_GLIBCXX_USE_CXX11_ABI=0"

2.5 Compile

# 如果 GitHub 不通，替换 workspace 文件中的 github.com URL：
sed -i 's|"https://github.com/|"https://ghfast.top/https://github.com/|g' \
    tensorflow/workspace2.bzl

bazel build --config=opt \
    //tensorflow/tools/pip_package:build_pip_package \
    --jobs=$(nproc)

# 打包 wheel
./bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tf_whl

# 安装
pip install /tmp/tf_whl/tensorflow-2.6.5-*.whl --no-deps

2.6 Install Supporting Dependencies

pip install 'absl-py~=0.10' 'clang~=5.0' 'flatbuffers~=1.12.0' \
    'keras~=2.6' 'keras-preprocessing~=1.1.1' 'opt-einsum~=3.3.0' \
    'six~=1.15.0' 'termcolor~=1.1.0' 'wrapt~=1.12.1' \
    'grpcio' 'protobuf' 'tensorboard~=2.6' 'tensorflow-estimator~=2.6' \
    'typing-extensions>=3.7,<3.11' 'google-pasta' 'astunparse'

# 验证
python -c "import tensorflow as tf; print(tf.__version__); \
    print(tf.reduce_sum(tf.random.normal([100,100])))"

---

Step 3: Install tfplugin (npu_device)

3.1 Download Ascend-cann-tfplugin

# CANN 8.0.RC3 版本（与 TF 2.6 ABI 匹配）
wget 'https://ascend-repo.obs.cn-east-2.myhuaweicloud.com/CANN/CANN%208.0.RC3/Ascend-cann-tfplugin_8.0.RC3_linux-aarch64.run' \
     -O /tmp/tfplugin.run
chmod +x /tmp/tfplugin.run

3.2 Extract and Install the npu_device wheel

# 第一层解包
/tmp/tfplugin.run --noexec --extract=/tmp/tfplugin_ext

# 第二层解包
chmod +x /tmp/tfplugin_ext/run_package/CANN-fwkplugin-*.run
/tmp/tfplugin_ext/run_package/CANN-fwkplugin-*.run --noexec --extract=/tmp/fwk_ext

# 安装 npu_device wheel
pip install /tmp/fwk_ext/fwkplugin/bin/npu_device-0.1-py3-none-any.whl

3.3 Install CANN Python Dependencies

pip install decorator sympy scipy attrs psutil

3.4 Verify NPU Device

source /home/Ascend/8.5/cann-8.5.0/set_env.sh  # 根据实际路径

python -c "
import npu_device
npu = npu_device.open().as_default()
print('NPU device opened')
import tensorflow as tf
print('result:', tf.reduce_sum(tf.constant([1.0, 2.0, 3.0])).numpy())
"
# 输出 result: 6.0 表示成功

---

Step 4: Code Adaptation

4.1 Minimally Invasive Adaptation (Recommended)

For most TF 2.x projects, simply add two lines at the top of the entry script:

import npu_device
npu_device.open().as_default()

TF's standard Keras APIs (Dense, Conv, LSTM, BatchNorm, etc.) will automatically execute on the NPU.

4.2 Automatic Migration Tool convert_tf2npu (Optional)

CANN provides the convert_tf2npu tool for batch code conversion:

# 工具位置（以 CANN 8.5 为例）
TOOL=/home/Ascend/8.5/cann-8.5.0/tools/ms_fmk_transplt

# 也可在 tensorflow-1.15-ascend 仓库中找到
TOOL=/home/show/tensorflow-1.15-ascend/convert_tf2npu

# 运行（TF 2.x 项目使用 v2 配置）
cd $TOOL
python main.py -i /path/to/project -o /path/to/output -m /path/to/main.py

Note: This tool mainly handles adaptation for TF 1.x patterns such as tf.Session、tf.ConfigProto . For pure TF 2.x Eager mode projects, usually manually adding npu_device.open() is sufficient.

4.3 Common Adaptation Points

Original Code	Adaptation Method
import tensorflow as tf	Add before import npu_device; npu_device.open().as_default()
tf.device('/gpu:0')	Delete or change to tf.device('/cpu:0')(npu_device automatically takes over the default device)
tensorflow-gpu dependencies	Replace with source-compiled tensorflow
tf.config.experimental.set_memory_growth(gpu, True)	Delete (not needed for NPU)
CUDA-related environment variables	No longer needed, switch to ASCEND_RT_VISIBLE_DEVICES

---

Step 5: Accuracy Verification

CPU vs NPU Comparison Template

import json, numpy as np

# 加载 CPU 和 NPU 的预测结果
with open('cpu_pred_scores.json') as f:
    cpu = json.load(f)
with open('npu_pred_scores.json') as f:
    npu = json.load(f)

cpu_scores = np.array(cpu['Y_hat'])
npu_scores = np.array(npu['Y_hat'])

max_diff = np.max(np.abs(cpu_scores - npu_scores))
mean_diff = np.mean(np.abs(cpu_scores - npu_scores))
cos_sim = np.dot(cpu_scores.flatten(), npu_scores.flatten()) / (
    np.linalg.norm(cpu_scores) * np.linalg.norm(npu_scores) + 1e-12)

print(f"max_diff={max_diff:.6e}")
print(f"mean_diff={mean_diff:.6e}")
print(f"cosine_sim={cos_sim:.8f}")

Expected Accuracy Range

Precision Mode	Typical max_diff	Description
allow_fp32	< 1e-5	Pure FP32, highest precision
allow_hf32 (default)	< 1e-2	Uses HF32 mixed precision, better performance

If higher precision is needed:

import npu_device
npu_device.global_options().precision_mode = "allow_fp32"
npu_device.open().as_default()

---

Pitfall Log

Compilation-related

1. No pip wheel for aarch64: TF 2.6.5 has no precompiled pip package on aarch64, so it must be compiled from source
2. ABI must be 0: The _GLIBCXX_USE_CXX11_ABI of TF and tfplugin must be consistent (both 0)
3. nsync patch: nsync atomic operations on aarch64 require adding __sync_synchronize() barriers
4. GitHub timeout: GitHub may time out when bazel downloads dependencies; use sed to replace with a mirror (ghfast.top)
5. Bazel version: TF 2.6.5 strictly requires Bazel 3.7.2; too high or too low will fail

Runtime-related

1. npu_device missing Python dependencies：decorator、sympy Missing modules such as will cause GE engine initialization to fail
2. numpy version: TF 2.6 is incompatible with numpy 1.24+ (np.object has been removed), use 1.21.x instead
3. h5py version: TF 2.6 requires h5py~=3.1.0, but 3.7.0 also works; use --no-deps to install TF and bypass it
4. CANN version compatibility: npu_device from tfplugin 8.0.RC3 works normally in the CANN 8.5 environment

Accuracy-related

1. HF32 mode: Ascend910 uses HF32 (half-float32) by default; some operators have lower precision than pure FP32
2. Unrecognized graph engine option warning: Caused by CANN version differences, does not affect correctness

---

Known Limitations

• Only supports TF 2.6.5 (bound to tfplugin ABI)
• aarch64 must compile TF from source (about 4-5 minutes, depending on the number of cores)
• Does not support tf.distribute multi-card strategy (must use npu_device's dedicated distributed API)
• Some dynamic shape operators may fall back to CPU execution

Companion Scripts

• TensorFlow Community environment check:python scripts/check_tf_npu_env.py --open-device

References

• TF Community Compilation and Installation Checklist:references/build-checklist.md