Ray is a unified framework for scaling AI and Python applications. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

A flexible, high-performance serving system for machine learning models

AI + Data, online. https://vespa.ai

In this repository, I will share some useful notes and references about deploying deep learning-based models in production.

An MLOps framework to package, deploy, monitor and manage thousands of production machine learning models

Serve, optimize and scale PyTorch models in production

⚡️An Easy-to-use and Fast Deep Learning Model Deployment Toolkit for ☁️Cloud 📱Mobile and 📹Edge. Including Image, Video, Text and Audio 20+ main stream scenarios and 150+ SOTA models with end-to-end optimization, multi-platform and multi-framework support.

Database system for AI-powered apps

TensorFlow template application for deep learning

DELTA is a deep learning based natural language and speech processing platform.

A comprehensive guide to building RAG-based LLM applications for production.

RayLLM - LLMs on Ray

A flexible, high-performance carrier for machine learning models（『飞桨』服务化部署框架）

Generic and easy-to-use serving service for machine learning models

A scalable inference server for models optimized with OpenVINO™

A unified end-to-end machine intelligence platform

Python + Inference - Model Deployment library in Python. Simplest model inference server ever.

Lineage metadata API, artifacts streams, sandbox, API, and spaces for Polyaxon

ML pipeline orchestration and model deployments on Kubernetes.

부스트캠프 AI Tech - Product Serving 자료