Generative AI reference workflows optimized for accelerated infrastructure and microservice architecture.

Add bisenetv2. My implementation of BiSeNet

This repository deploys YOLOv4 as an optimized TensorRT engine to Triton Inference Server

ClearML - Model-Serving Orchestration and Repository Solution

An Alternative for Triton Inference Server. Boosting DL Service Throughput 1.5-4x by Ensemble Pipeline Serving with Concurrent CUDA Streams for PyTorch/LibTorch Frontend and TensorRT/CVCUDA, etc., Backends

Deploy stable diffusion model with onnx/tenorrt + tritonserver

The Triton backend for the ONNX Runtime.

Hardware-accelerated DNN model inference ROS 2 packages using NVIDIA Triton/TensorRT for both Jetson and x86_64 with CUDA-capable GPU

Deploy DL/ ML inference pipelines with minimal extra code.

OpenAI compatible API for TensorRT LLM triton backend

Set up CI in DL/ cuda/ cudnn/ TensorRT/ onnx2trt/ onnxruntime/ onnxsim/ Pytorch/ Triton-Inference-Server/ Bazel/ Tesseract/ PaddleOCR/ NVIDIA-docker/ minIO/ Supervisord on AGX or PC from scratch.

Compare multiple optimization methods on triton to imporve model service performance

Tiny configuration for Triton Inference Server

Build Recommender System with PyTorch + Redis + Elasticsearch + Feast + Triton + Flask. Vector Recall, DeepFM Ranking and Web Application.

Advanced inference pipeline using NVIDIA Triton Inference Server for CRAFT Text detection (Pytorch), included converter from Pytorch -> ONNX -> TensorRT, Inference pipelines (TensorRT, Triton server - multi-format). Supported model format for Triton inference: TensorRT engine, Torchscript, ONNX

triton server ensemble model demo

Diffusion Model for Voice Conversion

Provides an ensemble model to deploy a YoloV8 ONNX model to Triton

Serving Example of CodeGen-350M-Mono-GPTJ on Triton Inference Server with Docker and Kubernetes

Magface Triton Inferece Server Using Tensorrt