TensorFold

exllamav2

A fast inference library for running LLMs locally on modern consumer-class GPUs

github.comLibraryOpen sourcelatest reaction 3 years ago

exllamav3

An optimized quantization and inference library for running LLMs locally on modern consumer-class GPUs

github.comLibraryOpen sourcelatest reaction last year

sonar

Large-scale LLM inference engine

github.comOpen sourcelatest reaction 3 years ago

Open-source AMDGCN kernels for optimizing LLM inference

github.comOpen source2 reactionslatest reaction 2 months ago

1Cat-vLLM

V100 / SM70-focused vLLM engineering fork for modern LLM inference.

github.comOpen sourcelatest reaction 8 months ago