Jammy@Jetson Orin Nano - Tensorflow GPU版本安装

1. 源由
2. 问题
- 2.1 Tensorflow跑以下示例代码的时候，发现`jtop`中6个CPU占用率都跑满了。
- 2.2 Jetson Orin Nano运行Tensorflow示例结果不一致
3. 分析
- 3.1 当前版本Tensorflow 2.16.1
- 3.2 GPU版本二进制安装
- 3.3 GPU版本源代码安装
- - 3.3.1 问题1 ERROR: no such target '//tensorflow/tools/pip_package:wheel'
  - 3.3.2 问题2 fatal error: 'cstddef' file not found
  - 3.3.3 问题3 clang: error: unsupported CUDA gpu architecture: sm_90
4. 总结
5. 参考资料

1. 源由

前面关于Jetson Orin Nano板子的软件安装已经总结了不少，不过这个板子最大的好处是GPU的运算能力，比如：《ubuntu22.04@Jetson Orin Nano之OpenCV安装》。

不过最近发现目前安装的tensorflow 2.16.1版本，在做运算时，压根没有用到GPU，而是在大量的使用CPU计算。这个就有点郁闷了，到底原因出在哪里？必须Fix~~

2. 问题

2.1 Tensorflow跑以下示例代码的时候，发现`jtop`中6个CPU占用率都跑满了。

《Jammy@Jetson Orin - Tensorflow & Keras Get Started: 004 Keras Pre-Trained ImageNet Models》
《Jammy@Jetson Orin - Tensorflow & Keras Get Started: 005 Keras Fine Tune Pre-Trained Models GTSRB》

显然，Jetson Orin Nano是满满的可以跑GPU的，怎么到CPU上去运算了？

初步怀疑就是Tensorflow版本没有支持NVIDIA的GPU。

2.2 Jetson Orin Nano运行Tensorflow示例结果不一致

非常奇怪的一个问题，Tensorflow示例代码-transfer_learning.ipynb在Jetson Orin Nano上运行的结果与其文档中的展示结果不一致，而Colab上运行是一致的。

从逻辑的角度看，这个不应该出现这种不一致，难道是这个示例要求非常高的内存资源？？？目前，NVIDIA反馈：Jetson Orin的CPU和GPU是共用内存的，因此，内存资源是一个瓶颈，建议大家买8GB以上，最好16GB的。

Inconsistency of NVIDIA 2.15.0+nv24.03 v.s. Colab v.s. Tensorflow Documentation
Doc(Transfer learning and fine-tuning) is quite different from real executive result. #66696

3. 分析

3.1 当前版本Tensorflow 2.16.1

通过之前安装命令，以及Tensorflow官网安装的信息看，当前安装的版本应该只是CPU版本，并非GPU的版本。

Jammy@Jetson Orin - Tensorflow & Keras Get Started: 000 setup for tutorial
Tensorflow - Install TensorFlow with pip

3.2 GPU版本二进制安装

$ sudo pip3 install --extra-index-url https://developer.download.nvidia.com/compute/redist/jp/v60dp tensorflow==2.15.0+nv24.03

How to install tensorflow with GPU support on Jetson Orin Nano?
Installing TensorFlow for Jetson Platform

当前JetPack 6.0DP 支持的Tensorflow版本，暂不支持2.16.1版本
在这里插入图片描述

3.3 GPU版本源代码安装

为了安装Tensorflow 2.16.1最新发布稳定版本，只有最后的一个选择，从源代码编译。

Tensorflow - Build from source
How to compile tensorflow for Jetson Orin Nano?
tensorflow was not compiled for cuda support jetson nano ??

在Linux 36.2@Jetson Orin Nano之基础环境构建基础上建立编译环境：

$ sudo apt-get install bazel-bootstrap
$ sudo apt-get install clang
$ sudo apt-get install python3-clang

$ cd ~/Downloads
$ wget https://github.com/bazelbuild/bazelisk/releases/download/v1.8.1/bazelisk-linux-arm64
$ chmod +x bazelisk-linux-arm64
$ sudo mv bazelisk-linux-arm64 /usr/local/bin/bazel
$ which bazel
$ /usr/local/bin/bazel

$ export TF_PYTHON_VERSION=3.10

目前上无法顺利编译通过，请持续关注：Tensorflow v2.16.1 GPU version local build on Jetson Orin Nano failed

NVIDIA支持的反馈，Tensorflow的步骤与官网一致，至于遇到的问题，需要去和Tensorflow进一步沟通。NVIDIA不做支持。

在这里插入图片描述

3.3.1 问题1 ERROR: no such target ‘//tensorflow/tools/pip_package:wheel’

根据官网指南，执行报错：ERROR: no such target ‘//tensorflow/tools/pip_package:wheel’

$ bazel build //tensorflow/tools/pip_package:wheel --repo_env=WHEEL_NAME=tensorflow --config=cuda
... ...
WARNING: The following configs were expanded more than once: [tensorrt, cuda_clang, cuda]. For repeatable flags, repeats are counted twice and may lead to unexpected behavior.
ERROR: Skipping '//tensorflow/tools/pip_package:wheel': no such target '//tensorflow/tools/pip_package:wheel': target 'wheel' not declared in package 'tensorflow/tools/pip_package' defined by /home/daniel/OpenCV/tensorflow/tensorflow/tools/pip_package/BUILD (Tip: use `query "//tensorflow/tools/pip_package:*"` to see all the targets in that package)
WARNING: Target pattern parsing failed.
ERROR: no such target '//tensorflow/tools/pip_package:wheel': target 'wheel' not declared in package 'tensorflow/tools/pip_package' defined by /home/daniel/OpenCV/tensorflow/tensorflow/tools/pip_package/BUILD (Tip: use `query "//tensorflow/tools/pip_package:*"` to see all the targets in that package)
INFO: Elapsed time: 1.498s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded)

调整编译目标：build_pip_package

$ bazel query "//tensorflow/tools/pip_package:*"
//tensorflow/tools/pip_package:BUILD
//tensorflow/tools/pip_package:MANIFEST.in
//tensorflow/tools/pip_package:README
//tensorflow/tools/pip_package:THIRD_PARTY_NOTICES.txt
//tensorflow/tools/pip_package:build_pip_package
//tensorflow/tools/pip_package:build_pip_package.sh
//tensorflow/tools/pip_package:included_headers
//tensorflow/tools/pip_package:included_headers_gather
//tensorflow/tools/pip_package:licenses
//tensorflow/tools/pip_package:setup.py
//tensorflow/tools/pip_package:simple_console
//tensorflow/tools/pip_package:simple_console.py
//tensorflow/tools/pip_package:xla_build/CMakeLists.txt
//tensorflow/tools/pip_package:xla_cmake
//tensorflow/tools/pip_package:xla_compiled_cpu_runtime_srcs.txt
//tensorflow/tools/pip_package:xla_compiled_cpu_runtime_srcs.txt_file
Loading: 0 packages loaded
$ bazel build //tensorflow/tools/pip_package:build_pip_package --repo_env=WHEEL_NAME=tensorflow --config=cuda

3.3.2 问题2 fatal error: ‘cstddef’ file not found

编译报错，头文件找不到

$ bazel build //tensorflow/tools/pip_package:build_pip_package --repo_env=WHEEL_NAME=tensorflow --config=cuda
... ...
WARNING: The following configs were expanded more than once: [tensorrt, cuda_clang, cuda]. For repeatable flags, repeats are counted twice and may lead to unexpected behavior.
INFO: Analyzed target //tensorflow/tools/pip_package:build_pip_package (704 packages loaded, 50634 targets configured).
INFO: Found 1 target...
ERROR: /home/daniel/.cache/bazel/_bazel_daniel/11588ef030db288b35dd97b3c9d34cbc/external/llvm-project/llvm/BUILD.bazel:191:11: Compiling llvm/lib/Demangle/RustDemangle.cpp failed: (Exit 1): clang failed: error executing command (from target @llvm-project//llvm:Demangle) /usr/lib/llvm-14/bin/clang -MD -MF bazel-out/aarch64-opt/bin/external/llvm-project/llvm/_objs/Demangle/RustDemangle.pic.d ... (remaining 85 arguments skipped)
In file included from external/llvm-project/llvm/lib/Demangle/RustDemangle.cpp:14:
external/llvm-project/llvm/include/llvm/Demangle/Demangle.h:12:10: fatal error: 'cstddef' file not found
#include <cstddef>
         ^~~~~~~~~
1 error generated.
Target //tensorflow/tools/pip_package:build_pip_package failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 731.518s, Critical Path: 0.46s
INFO: 24 processes: 17 internal, 7 local.
FAILED: Build did NOT complete successfully

安装· libstdc+±12-dev·库

$ sudo apt install libstdc++-12-dev

3.3.3 问题3 clang: error: unsupported CUDA gpu architecture: sm_90

Tensorflow v2.16.1 GPU version local build on Jetson Orin Nano failed

$ bazel build //tensorflow/tools/pip_package:build_pip_package --repo_env=WHEEL_NAME=tensorflow --config=cuda
... ...
ERROR: /home/daniel/.cache/bazel/_bazel_daniel/11588ef030db288b35dd97b3c9d34cbc/external/local_xla/xla/stream_executor/cuda/BUILD:505:13: Compiling xla/stream_executor/cuda/cuda_conditional_kernels.cu.cc failed: (Exit 1): clang failed: error executing command (from target @local_xla//xla/stream_executor/cuda:cuda_conditional_kernels) /usr/lib/llvm-14/bin/clang -MD -MF bazel-out/aarch64-opt/bin/external/local_xla/xla/stream_executor/cuda/_objs/cuda_conditional_kernels/cuda_conditional_kernels.cu.pic.d ... (remaining 72 arguments skipped)
clang: warning: CUDA version is newer than the latest supported version 11.5 [-Wunknown-cuda-version]
clang: error: unsupported CUDA gpu architecture: sm_90
Target //tensorflow/tools/pip_package:build_pip_package failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 853.288s, Critical Path: 61.65s
INFO: 11696 processes: 8427 internal, 3269 local.
FAILED: Build did NOT complete successfully

4. 总结

小白入手，通常都是卡在这种没有技术含量的事情上，而这些会导致真正核心技术的发展。

希望通过这些简单的总结，帮助到各位希望学习技术的朋友，在后面的技术道路上走的更顺畅，减少这些由于不熟悉导致的浪费时间。把重点放在技术的学习、突破和创新上。

另外，不同版本的代码之间可能存在或多或少的兼容性问题。

Multiple executive warnings after switching tensorflow from 2.16.1 CPU to v60dp tensorflow==2.15.0+nv24.03 GPU version

这也是在技术管理上最重要的一环：版本管理。

5. 参考资料

【1】Linux 36.2@Jetson Orin Nano之基础环境构建
【2】Linux 36.2@Jetson Orin Nano之Hello AI World！
【3】ubuntu22.04@Jetson Orin Nano之OpenCV安装
【3】ubuntu22.04@Jetson Orin Nano之CSI IMX219安装
【3】ubuntu22.04@Jetson Orin Nano安装&配置VNC服务端
【3】Jammy@Jetson Orin - Tensorflow & Keras Get Started: 000 setup for tutorial