TensorFold

exllamav2

A fast inference library for running LLMs locally on modern consumer-class GPUs

github.com라이브러리오픈소스최근 반응 3년 전

exllamav3

An optimized quantization and inference library for running LLMs locally on modern consumer-class GPUs

github.com라이브러리오픈소스최근 반응 작년

sonar

Large-scale LLM inference engine

github.com오픈소스최근 반응 3년 전

Open-source AMDGCN kernels for optimizing LLM inference

github.com오픈소스반응 2개최근 반응 2개월 전

1Cat-vLLM

V100 / SM70-focused vLLM engineering fork for modern LLM inference.

github.com오픈소스최근 반응 8개월 전