1. 源由
前面关于Jetson Orin Nano板子的软件安装已经总结了不少，不过这个板子最大的好处是GPU的运算能力，比如：《ubuntu22.04@Jetson Orin Nano之OpenCV安装》。
不过最近发现目前安装的tensorflow 2.16.1版本，在做运算时，压根没有用到GPU，而是在大量的使用CPU计算。这个就有点郁闷了，到底原因出在哪里？必须Fix~~
2. 问题
2.1 Tensorflow跑以下示例代码的时候，发现jtop中6个CPU占用率都跑满了。
- 《Jammy@Jetson Orin - Tensorflow & Keras Get Started: 004 Keras Pre-Trained ImageNet Models》
- 《Jammy@Jetson Orin - Tensorflow & Keras Get Started: 005 Keras Fine Tune Pre-Trained Models GTSRB》
显然，Jetson Orin Nano是满满的可以跑GPU的，怎么到CPU上去运算了？
初步怀疑就是Tensorflow版本没有支持NVIDIA的GPU。
2.2 Jetson Orin Nano运行Tensorflow示例结果不一致
非常奇怪的一个问题，Tensorflow示例代码-transfer_learning.ipynb在Jetson Orin Nano上运行的结果与其文档中的展示结果不一致，而Colab上运行是一致的。
从逻辑的角度看，这个不应该出现这种不一致，难道是这个示例要求非常高的内存资源？？？目前，NVIDIA反馈：Jetson Orin的CPU和GPU是共用内存的，因此，内存资源是一个瓶颈，建议大家买8GB以上，最好16GB的。
- Inconsistency of NVIDIA 2.15.0+nv24.03 v.s. Colab v.s. Tensorflow Documentation
- Doc(Transfer learning and fine-tuning) is quite different from real executive result. #66696
3. 分析
3.1 当前版本Tensorflow 2.16.1
通过之前安装命令，以及Tensorflow官网安装的信息看，当前安装的版本应该只是CPU版本，并非GPU的版本。
- Jammy@Jetson Orin - Tensorflow & Keras Get Started: 000 setup for tutorial
- Tensorflow - Install TensorFlow with pip
3.2 GPU版本二进制安装
$ sudo pip3 install --extra-index-url https://developer.download.nvidia.com/compute/redist/jp/v60dp tensorflow==2.15.0+nv24.03
- How to install tensorflow with GPU support on Jetson Orin Nano?
- Installing TensorFlow for Jetson Platform
当前JetPack 6.0DP 支持的Tensorflow版本，暂不支持2.16.1版本
3.3 GPU版本源代码安装
为了安装Tensorflow 2.16.1最新发布稳定版本，只有最后的一个选择，从源代码编译。
- Tensorflow - Build from source
- How to compile tensorflow for Jetson Orin Nano?
- tensorflow was not compiled for cuda support jetson nano ??
在Linux 36.2@Jetson Orin Nano之基础环境构建基础上建立编译环境：
$ sudo apt-get install bazel-bootstrap
$ sudo apt-get install clang
$ sudo apt-get install python3-clang
$ cd ~/Downloads
$ wget https://github.com/bazelbuild/bazelisk/releases/download/v1.8.1/bazelisk-linux-arm64
$ chmod +x bazelisk-linux-arm64
$ sudo mv bazelisk-linux-arm64 /usr/local/bin/bazel
$ which bazel
$ /usr/local/bin/bazel
$ export TF_PYTHON_VERSION=3.10
目前上无法顺利编译通过，请持续关注：Tensorflow v2.16.1 GPU version local build on Jetson Orin Nano failed
NVIDIA支持的反馈，Tensorflow的步骤与官网一致，至于遇到的问题，需要去和Tensorflow进一步沟通。NVIDIA不做支持。
3.3.1 问题1 ERROR: no such target ‘//tensorflow/tools/pip_package:wheel’
根据官网指南，执行报错：ERROR: no such target ‘//tensorflow/tools/pip_package:wheel’
$ bazel build //tensorflow/tools/pip_package:wheel --repo_env=WHEEL_NAME=tensorflow --config=cuda
... ...
WARNING: The following configs were expanded more than once: [tensorrt, cuda_clang, cuda]. For repeatable flags, repeats are counted twice and may lead to unexpected behavior.
ERROR: Skipping '//tensorflow/tools/pip_package:wheel': no such target '//tensorflow/tools/pip_package:wheel': target 'wheel' not declared in package 'tensorflow/tools/pip_package' defined by /home/daniel/OpenCV/tensorflow/tensorflow/tools/pip_package/BUILD (Tip: use `query "//tensorflow/tools/pip_package:*"` to see all the targets in that package)
WARNING: Target pattern parsing failed.
ERROR: no such target '//tensorflow/tools/pip_package:wheel': target 'wheel' not declared in package 'tensorflow/tools/pip_package' defined by /home/daniel/OpenCV/tensorflow/tensorflow/tools/pip_package/BUILD (Tip: use `query "//tensorflow/tools/pip_package:*"` to see all the targets in that package)
INFO: Elapsed time: 1.498s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded)
调整编译目标：build_pip_package
$ bazel query "//tensorflow/tools/pip_package:*"
//tensorflow/tools/pip_package:BUILD
//tensorflow/tools/pip_package:MANIFEST.in
//tensorflow/tools/pip_package:README
//tensorflow/tools/pip_package:THIRD_PARTY_NOTICES.txt
//tensorflow/tools/pip_package:build_pip_package
//tensorflow/tools/pip_package:build_pip_package.sh
//tensorflow/tools/pip_package:included_headers
//tensorflow/tools/pip_package:included_headers_gather
//tensorflow/tools/pip_package:licenses
//tensorflow/tools/pip_package:setup.py
//tensorflow/tools/pip_package:simple_console
//tensorflow/tools/pip_package:simple_console.py
//tensorflow/tools/pip_package:xla_build/CMakeLists.txt
//tensorflow/tools/pip_package:xla_cmake
//tensorflow/tools/pip_package:xla_compiled_cpu_runtime_srcs.txt
//tensorflow/tools/pip_package:xla_compiled_cpu_runtime_srcs.txt_file
Loading: 0 packages loaded
$ bazel build //tensorflow/tools/pip_package:build_pip_package --repo_env=WHEEL_NAME=tensorflow --config=cuda
3.3.2 问题2 fatal error: ‘cstddef’ file not found
编译报错，头文件找不到
$ bazel build //tensorflow/tools/pip_package:build_pip_package --repo_env=WHEEL_NAME=tensorflow --config=cuda
... ...
WARNING: The following configs were expanded more than once: [tensorrt, cuda_clang, cuda]. For repeatable flags, repeats are counted twice and may lead to unexpected behavior.
INFO: Analyzed target //tensorflow/tools/pip_package:build_pip_package (704 packages loaded, 50634 targets configured).
INFO: Found 1 target...
ERROR: /home/daniel/.cache/bazel/_bazel_daniel/11588ef030db288b35dd97b3c9d34cbc/external/llvm-project/llvm/BUILD.bazel:191:11: Compiling llvm/lib/Demangle/RustDemangle.cpp failed: (Exit 1): clang failed: error executing command (from target @llvm-project//llvm:Demangle) /usr/lib/llvm-14/bin/clang -MD -MF bazel-out/aarch64-opt/bin/external/llvm-project/llvm/_objs/Demangle/RustDemangle.pic.d ... (remaining 85 arguments skipped)
In file included from external/llvm-project/llvm/lib/Demangle/RustDemangle.cpp:14:
external/llvm-project/llvm/include/llvm/Demangle/Demangle.h:12:10: fatal error: 'cstddef' file not found
#include <cstddef>
^~~~~~~~~
1 error generated.
Target //tensorflow/tools/pip_package:build_pip_package failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 731.518s, Critical Path: 0.46s
INFO: 24 processes: 17 internal, 7 local.
FAILED: Build did NOT complete successfully
安装· libstdc+±12-dev·库
$ sudo apt install libstdc++-12-dev
3.3.3 问题3 clang: error: unsupported CUDA gpu architecture: sm_90
$ bazel build //tensorflow/tools/pip_package:build_pip_package --repo_env=WHEEL_NAME=tensorflow --config=cuda
... ...
ERROR: /home/daniel/.cache/bazel/_bazel_daniel/11588ef030db288b35dd97b3c9d34cbc/external/local_xla/xla/stream_executor/cuda/BUILD:505:13: Compiling xla/stream_executor/cuda/cuda_conditional_kernels.cu.cc failed: (Exit 1): clang failed: error executing command (from target @local_xla//xla/stream_executor/cuda:cuda_conditional_kernels) /usr/lib/llvm-14/bin/clang -MD -MF bazel-out/aarch64-opt/bin/external/local_xla/xla/stream_executor/cuda/_objs/cuda_conditional_kernels/cuda_conditional_kernels.cu.pic.d ... (remaining 72 arguments skipped)
clang: warning: CUDA version is newer than the latest supported version 11.5 [-Wunknown-cuda-version]
clang: error: unsupported CUDA gpu architecture: sm_90
Target //tensorflow/tools/pip_package:build_pip_package failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 853.288s, Critical Path: 61.65s
INFO: 11696 processes: 8427 internal, 3269 local.
FAILED: Build did NOT complete successfully
4. 总结
小白入手，通常都是卡在这种没有技术含量的事情上，而这些会导致真正核心技术的发展。
希望通过这些简单的总结，帮助到各位希望学习技术的朋友，在后面的技术道路上走的更顺畅，减少这些由于不熟悉导致的浪费时间。把重点放在技术的学习、突破和创新上。
另外，不同版本的代码之间可能存在或多或少的兼容性问题。
这也是在技术管理上最重要的一环：版本管理。
经过将近一个月的折腾，我们找到了原因：Jetson Orin Nano Linux 36.2 6.0DP 对tensorflow支持上存在BUG，导致某些场景异常。不推荐使用6.0DP, Develop View Version
- Linux 36.2@Jetson Orin Nano之基础环境构建
- Inconsistency of NVIDIA 2.15.0+nv24.03 v.s. Colab v.s. Tensorflow Documentation
最终solution：Jetson Orin Nano v6.0 + tensorflow2.15.0+nv24.05 GPU版本安装
5. 参考资料
【1】Linux 36.2@Jetson Orin Nano之基础环境构建
【2】Linux 36.2@Jetson Orin Nano之Hello AI World！
【3】ubuntu22.04@Jetson Orin Nano之OpenCV安装
【4】ubuntu22.04@Jetson Orin Nano之CSI IMX219安装
【5】ubuntu22.04@Jetson Orin Nano安装&配置VNC服务端
【6】Jammy@Jetson Orin - Tensorflow & Keras Get Started: 000 setup for tutorial