Pinned

1. lm-sys/FastChat lm-sys/FastChat Public

   An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.

   Python 33.9k 4.1k

2. alpa-projects/alpa alpa-projects/alpa Public

   Training and serving large-scale neural networks with auto parallelization.

   Python 3k 341

3. sgl-project/sglang sgl-project/sglang Public

   SGLang is a structured generation language designed for large language models (LLMs). It makes your interaction with models faster and more controllable.

   Python 2.3k 140

4. FMInference/FlexGen FMInference/FlexGen Public

   Running large language models on a single GPU for throughput-oriented scenarios.

   Python 9k 522

5. apache/tvm apache/tvm Public

   Open deep learning compiler stack for cpu, gpu and specialized accelerators

   Python 11.2k 3.3k

6. vllm-project/vllm vllm-project/vllm Public

   A high-throughput and memory-efficient inference and serving engine for LLMs

   Python 18.5k 2.4k