← Trending

Tangerine Index lists products launched since ChatGPT and people's reactions to them.

exllamav2

github.com

A fast inference library for running LLMs locally on modern consumer-class GPUs

LibraryOpen source

Visit github.comexternal siteGitHub4,651 starsexternal site

Reactions (1)

Similar products

by description similarity

  1. exllamav3

    An optimized quantization and inference library for running LLMs locally on modern consumer-class GPUs

    github.comLibraryOpen sourcelatest reaction

    GitHub
    1,800 stars

  2. airllm

    AirLLM 2.8T LLM inference with single 4GB GPU

    github.comOpen sourcelatest reaction

    GitHub
    41,908 stars

  3. vllm

    A high-throughput and memory-efficient inference and serving engine for LLMs

    github.comOpen sourcelatest reaction

    GitHub
    96,511 stars

  4. TensorFold

    LLM Inference Engine for Metal, CUDA and Vulkan.

    github.comOpen sourcelatest reaction

    GitHub
    1,686 stars

  5. T-MAC

    Low-bit LLM inference on CPU/NPU with lookup table

    github.comOpen sourcelatest reaction

    GitHub
    1,005 stars