Fast inference engine for Transformer models

Tuned OpenCL BLAS

🎉CUDA 笔记 / 大模型手撕CUDA / C++笔记，更新随缘: flash_attn、sgemm、sgemv、warp reduce、block reduce、dot product、elementwise、softmax、layernorm、rmsnorm、hist etc.

BLISlab: A Sandbox for Optimizing GEMM

The HPC toolbox: fused matrix multiplication, convolution, data-parallel strided tensor primitives, OpenMP facilities, SIMD, JIT Assembler, CPU detection, state-of-the-art vectorized BLAS for floats and integers

Optimizing SGEMM kernel functions on NVIDIA GPUs to a close-to-cuBLAS performance.

Several optimization methods of half-precision general matrix multiplication (HGEMM) using tensor core with WMMA API and MMA PTX instruction.

Stretching GPU performance for GEMMs and tensor contractions.

DBCSR: Distributed Block Compressed Sparse Row matrix library

Single file libraries for C/C++

🔥🔥🔥 A collection of some awesome public NVIDIA CUDA, cuBLAS, cuDNN, TensorRT, AMD ROCm and FPGA projects.

Stepwise optimizations of DGEMM on CPU, reaching performance faster than Intel MKL eventually, even under multithreading.

FP64 equivalent GEMM via Int8 Tensor Cores using the Ozaki scheme

hipBLASLt is a library that provides general matrix-matrix operations with a flexible API and extends functionalities beyond a traditional BLAS library

Code for testing the native float16 matrix multiplication performance on Tesla P100 and V100 GPU based on cublasHgemm

Serial and parallel implementations of matrix multiplication

code for benchmarking GPU performance based on cublasSgemm and cublasHgemm

Specialized Parallel Linear Algebra, providing distributed GEMM functionality for specific matrix distributions with optional GPU acceleration.

PyTorch bindings for openai-gemm