Machine Learning Containers for Jetson and JetPack

Modular container build system that provides various AI/ML packages for NVIDIA Jetson 🚀🤖

ML	pytorch tensorflow onnxruntime deepstream tritonserver jupyterlab stable-diffusion
LLM	transformers text-generation-webui text-generation-inference llava llama.cpp exllama llamaspeak local_llm awq AutoGPTQ MiniGPT-4 MLC langchain optimum nemo
L4T	l4t-pytorch l4t-tensorflow l4t-ml l4t-diffusion l4t-text-generation
VIT	NanoOWL NanoSAM Segment Anything (SAM) Track Anything (TAM)
CUDA	cupy cuda-python pycuda numba cudf cuml
Robotics	ros ros2 opencv:cuda realsense zed
VectorDB	NanoDB FAISS RAFT
Audio	whisper whisperX riva audiocraft

See the packages directory for the full list, including pre-built container images and CI/CD status for JetPack/L4T.

Using the included tools, you can easily combine packages together for building your own containers. Want to run ROS2 with PyTorch and Transformers? No problem - just do the system setup, and build it on your Jetson like this:

$ ./build.sh --name=my_container pytorch transformers ros:humble-desktop

There are shortcuts for running containers too - this will pull or build a l4t-pytorch image that's compatible:

$ ./run.sh $(./autotag l4t-pytorch)

^{run.sh forwards arguments to docker run with some defaults added (like --runtime nvidia, mounts a /data cache, and detects devices)
autotag finds a container image that's compatible with your version of JetPack/L4T - either locally, pulled from a registry, or by building it.}

If you look at any package's readme (like l4t-pytorch), it will have detailed instructions for running it's container.

Documentation

• Package List
• Package Definitions
• System Setup
• Building Containers
• Running Containers

Check out the tutorials at the Jetson Generative AI Lab!

Getting Started

Refer to the System Setup page for tips about setting up your Docker daemon and memory/storage tuning.

sudo apt-get update && sudo apt-get install git python3-pip
git clone --depth=1 https://github.com/dusty-nv/jetson-containers
cd jetson-containers
pip3 install -r requirements.txt
./run.sh $(./autotag l4t-pytorch)

Or you can manually run a container image of your choice without using the helper scripts above:

sudo docker run --runtime nvidia -it --rm --network=host dustynv/l4t-pytorch:r35.4.1

Looking for the old jetson-containers? See the legacy branch.

Gallery

Multimodal Voice Chat with LLaVA-1.5 13B on NVIDIA Jetson AGX Orin (container: local_llm)

Interactive Voice Chat with Llama-2-70B on NVIDIA Jetson AGX Orin (container: local_llm)

Realtime Multimodal VectorDB on NVIDIA Jetson (container: nanodb)

NanoOWL - Open Vocabulary Object Detection ViT (container: nanoowl)

Live Llava on Jetson AGX Orin (container: local_llm)