Linux 36.2@Jetson Orin Nano之Hello AI World！
- 1. 源由
- 2. Hello AI World！
- 3. 步骤
- 3.1 准备阶段
- 3.2 获取代码
- 3.3 Python环境
- 3.4 重点环节
- 3.5 软件配置
- 3.6 PyTorch安装
- 3.7 编译链接
- 3.8 安装更新
- 4. 测试
- 4.1 video-viewer
- 4.2 detectnet
- 4.3 演示命令
- 5. 参考资料
- 6. 附录 AI模型
1. 源由
AI到底有多神奇？？？
记得神奇的年代有神奇的语言：“人有多大胆，地有多大产；不怕想不到，就怕做不到。“
暂且不去讨论这句话的背景，深意，以及各种解说。在这里，抓一个发散思维的要点，要能想，要感想！
好了，废话不多说，既然我们有了《Linux 36.2@Jetson Orin Nano基础环境构建》，就来看看用这些AI技术可以有些什么好玩的！
2. Hello AI World！
大体所有的新事物都会有个类似“Hello AI World”的介绍，让更加贴心的让我们快速接触和理解新事物。
- Linux应用程序之Helloworld入门
- ubuntu22.04@laptop OpenCV Get Started: 000_hello_opencv
这里也有一个Jetson AI的Hello AI World！。
大致有三种方法：
- Setting up Jetson with JetPack
- Running the Docker Container
- Building the Project from Source
通常来说，最难的就是从源代码来构建。因为程序对于环境的依赖关系，不是三言两语能够简单概括的。
3. 步骤
注：长城防火墙永远是技术的一种疼。遇到麻烦，请大家参考：Github操作网络异常笔记。
3.1 准备阶段
git用来获取最新github上的代码；而cmake主要用来做编译、链接的。
$ sudo apt-get update
$ sudo apt-get install git cmake
3.2 获取代码
获取最新的代码，通常是一个好的方法。不过也未必，最新不等于最好用。
不过我们的习惯是“不买合适的，不买最好的，就买最贵的；不用好用的，就用最新的。”
$ git clone https://github.com/dusty-nv/jetson-inference
$ cd jetson-inference
$ git submodule update --init
3.3 Python环境
Python在AI程序应用上是非常便捷的方法，当然讲效率那就去用C++。这里都Hello World，谁知道有没有Python示例代码。
$ sudo apt-get install libpython3-dev python3-numpy
3.4 重点环节
这里为什么说是重点，因为按照指南做，死活会出现各种编译、链接问题。经过笔者的牛刀小试，已经给各位解决了问题。
以下这些是Hello World必备的编译链接环境：
$ sudo apt-get install nvidia-cuda-dev tensorrt-dev nvidia-jetpack
3.5 软件配置
注：要按照笔者的方式进行CUDA_TOOLKIT_ROOT宏定义，切记！
$ cd jetson-inference # omit if working directory is already jetson-inference/ from above
$ mkdir build
$ cd build
$ cmake -D CUDA_TOOLKIT_ROOT=/usr/local/cuda ..
3.6 PyTorch安装
注：这个步骤好像并非必须，笔者就没有做。也许是用到的这个demo用不到吧。
$ cd jetson-inference/build
$ ./install-pytorch.sh
3.7 编译链接
$ cd jetson-inference/build # omit if working directory is already build/ from above
$ make -j$(nproc) # 多核编译，加快速度
3.8 安装更新
注：在没有完全搞清楚软件包安装路径前，不建议安装。其实在build目录下也可以用。
$ sudo make install
$ sudo ldconfig
编译好的应用程序都在aarch64下。
jetson-inference$ tree build/ -L 1
build/
├── aarch64
├── CMakeCache.txt
├── CMakeFiles
├── cmake_install.cmake
├── docs
├── examples
├── install-pytorch.rc
├── install-pytorch.sh
├── Makefile
├── python
├── tools
├── torch-2.1.0-cp310-cp310-linux_aarch64.whl
├── torchvision-310
└── utils
8 directories, 6 files
4. 测试
Jetson Orin Nano的板子用在AI上，最好的应用就是视频图像分析、物体跟踪。
不再献丑了，网上有大佬dusty-nv的讲座，大家自己看下：
S3E1 - Hello AI World Setup
不过，这种东西不过瘾，对吧。所以，我们先介绍两个常用的命令，弄个好玩的视频分析：
4.1 video-viewer
应用与视频的获取，比如：文件/RTP/RTSP/CSI/MIPI等等。
$ ./video-viewer --help
usage: video-viewer [--help] input_URI [output_URI]
View/output a video or image stream.
See below for additional arguments that may not be shown above.
positional arguments:
input_URI resource URI of input stream (see videoSource below)
output_URI resource URI of output stream (see videoOutput below)
videoSource arguments:
input resource URI of the input stream, for example:
* /dev/video0 (V4L2 camera #0)
* csi://0 (MIPI CSI camera #0)
* rtp://@:1234 (RTP stream)
* rtsp://user:pass@ip:1234 (RTSP stream)
* webrtc://@:1234/my_stream (WebRTC stream)
* file://my_image.jpg (image file)
* file://my_video.mp4 (video file)
* file://my_directory/ (directory of images)
--input-width=WIDTH explicitly request a width of the stream (optional)
--input-height=HEIGHT explicitly request a height of the stream (optional)
--input-rate=RATE explicitly request a framerate of the stream (optional)
--input-save=FILE path to video file for saving the input stream to disk
--input-codec=CODEC RTP requires the codec to be set, one of these:
* h264, h265
* vp8, vp9
* mpeg2, mpeg4
* mjpeg
--input-decoder=TYPE the decoder engine to use, one of these:
* cpu
* omx (aarch64/JetPack4 only)
* v4l2 (aarch64/JetPack5 only)
--input-flip=FLIP flip method to apply to input:
* none (default)
* counterclockwise
* rotate-180
* clockwise
* horizontal
* vertical
* upper-right-diagonal
* upper-left-diagonal
--input-loop=LOOP for file-based inputs, the number of loops to run:
* -1 = loop forever
* 0 = don't loop (default)
* >0 = set number of loops
videoOutput arguments:
output resource URI of the output stream, for example:
* file://my_image.jpg (image file)
* file://my_video.mp4 (video file)
* file://my_directory/ (directory of images)
* rtp://<remote-ip>:1234 (RTP stream)
* rtsp://@:8554/my_stream (RTSP stream)
* webrtc://@:1234/my_stream (WebRTC stream)
* display://0 (OpenGL window)
--output-codec=CODEC desired codec for compressed output streams:
* h264 (default), h265
* vp8, vp9
* mpeg2, mpeg4
* mjpeg
--output-encoder=TYPE the encoder engine to use, one of these:
* cpu
* omx (aarch64/JetPack4 only)
* v4l2 (aarch64/JetPack5 only)
--output-save=FILE path to a video file for saving the compressed stream
to disk, in addition to the primary output above
--bitrate=BITRATE desired target VBR bitrate for compressed streams,
in bits per second. The default is 4000000 (4 Mbps)
--headless don't create a default OpenGL GUI window
logging arguments:
--log-file=FILE output destination file (default is stdout)
--log-level=LEVEL message output threshold, one of the following:
* silent
* error
* warning
* success
* info
* verbose (default)
* debug
--verbose enable verbose logging (same as --log-level=verbose)
--debug enable debug logging (same as --log-level=debug)
4.2 detectnet
基于DNN的物体分析。
$ ./detectnet --help
usage: detectnet [--help] [--network=NETWORK] [--threshold=THRESHOLD] ...
input [output]
Locate objects in a video/image stream using an object detection DNN.
See below for additional arguments that may not be shown above.
positional arguments:
input resource URI of input stream (see videoSource below)
output resource URI of output stream (see videoOutput below)
detectNet arguments:
--network=NETWORK pre-trained model to load, one of the following:
* ssd-mobilenet-v1
* ssd-mobilenet-v2 (default)
* ssd-inception-v2
* peoplenet
* peoplenet-pruned
* dashcamnet
* trafficcamnet
* facedetect
--model=MODEL path to custom model to load (caffemodel, uff, or onnx)
--prototxt=PROTOTXT path to custom prototxt to load (for .caffemodel only)
--labels=LABELS path to text file containing the labels for each class
--input-blob=INPUT name of the input layer (default is 'data')
--output-cvg=COVERAGE name of the coverage/confidence output layer (default is 'coverage')
--output-bbox=BOXES name of the bounding output layer (default is 'bboxes')
--mean-pixel=PIXEL mean pixel value to subtract from input (default is 0.0)
--confidence=CONF minimum confidence threshold for detection (default is 0.5)
--clustering=CLUSTER minimum overlapping area threshold for clustering (default is 0.75)
--alpha=ALPHA overlay alpha blending value, range 0-255 (default: 120)
--overlay=OVERLAY detection overlay flags (e.g. --overlay=box,labels,conf)
valid combinations are: 'box', 'lines', 'labels', 'conf', 'none'
--profile enable layer profiling in TensorRT
objectTracker arguments:
--tracking flag to enable default tracker (IOU)
--tracker=TRACKER enable tracking with 'IOU' or 'KLT'
--tracker-min-frames=N the number of re-identified frames for a track to be considered valid (default: 3)
--tracker-drop-frames=N number of consecutive lost frames before a track is dropped (default: 15)
--tracker-overlap=N how much IOU overlap is required for a bounding box to be matched (default: 0.5)
videoSource arguments:
input resource URI of the input stream, for example:
* /dev/video0 (V4L2 camera #0)
* csi://0 (MIPI CSI camera #0)
* rtp://@:1234 (RTP stream)
* rtsp://user:pass@ip:1234 (RTSP stream)
* webrtc://@:1234/my_stream (WebRTC stream)
* file://my_image.jpg (image file)
* file://my_video.mp4 (video file)
* file://my_directory/ (directory of images)
--input-width=WIDTH explicitly request a width of the stream (optional)
--input-height=HEIGHT explicitly request a height of the stream (optional)
--input-rate=RATE explicitly request a framerate of the stream (optional)
--input-save=FILE path to video file for saving the input stream to disk
--input-codec=CODEC RTP requires the codec to be set, one of these:
* h264, h265
* vp8, vp9
* mpeg2, mpeg4
* mjpeg
--input-decoder=TYPE the decoder engine to use, one of these:
* cpu
* omx (aarch64/JetPack4 only)
* v4l2 (aarch64/JetPack5 only)
--input-flip=FLIP flip method to apply to input:
* none (default)
* counterclockwise
* rotate-180
* clockwise
* horizontal
* vertical
* upper-right-diagonal
* upper-left-diagonal
--input-loop=LOOP for file-based inputs, the number of loops to run:
* -1 = loop forever
* 0 = don't loop (default)
* >0 = set number of loops
videoOutput arguments:
output resource URI of the output stream, for example:
* file://my_image.jpg (image file)
* file://my_video.mp4 (video file)
* file://my_directory/ (directory of images)
* rtp://<remote-ip>:1234 (RTP stream)
* rtsp://@:8554/my_stream (RTSP stream)
* webrtc://@:1234/my_stream (WebRTC stream)
* display://0 (OpenGL window)
--output-codec=CODEC desired codec for compressed output streams:
* h264 (default), h265
* vp8, vp9
* mpeg2, mpeg4
* mjpeg
--output-encoder=TYPE the encoder engine to use, one of these:
* cpu
* omx (aarch64/JetPack4 only)
* v4l2 (aarch64/JetPack5 only)
--output-save=FILE path to a video file for saving the compressed stream
to disk, in addition to the primary output above
--bitrate=BITRATE desired target VBR bitrate for compressed streams,
in bits per second. The default is 4000000 (4 Mbps)
--headless don't create a default OpenGL GUI window
logging arguments:
--log-file=FILE output destination file (default is stdout)
--log-level=LEVEL message output threshold, one of the following:
* silent
* error
* warning
* success
* info
* verbose (default)
* debug
--verbose enable verbose logging (same as --log-level=verbose)
--debug enable debug logging (same as --log-level=debug)
4.3 演示命令
- 网络RTSP摄像头拉流&分析
$ cd jetson-inference/build
$ ./video-viewer --input-codec=h264 rtsp://192.168.78.201:8554/basesoci2c0muxi2c1ov564736
$ ./detectnet --input-codec=h264 rtsp://192.168.78.201:8554/basesoci2c0muxi2c1ov564736
- 视频文件播放&分析
$ cd jetson-inference/build
$ ./video-viewer --input-codec=h264 ../../../../TrackingBike.mp4
$ ./detectnet ../../../../TrackingBike.mp4 ../../../../TrackingBike_Detect.mp4
Extreme Mountain Biking FPV Drone Chasing
5. 参考资料
【1】Linux 36.2@Jetson Orin Nano基础环境构建
6. 附录 AI模型
detectNet 中有不少模型可供选择，默认情况: ssd-mobilenet-v2
- ssd-mobilenet-v1
- ssd-mobilenet-v2 (default)
- ssd-inception-v2
- peoplenet
- peoplenet-pruned
- dashcamnet
- trafficcamnet
- facedetect
$ cd jetson-inference/tools
$ ./download-models.sh
