← Trending

Tangerine Index lists products launched since ChatGPT and people's reactions to them.

exllamav3

github.com

An optimized quantization and inference library for running LLMs locally on modern consumer-class GPUs

LibraryOpen source

Visit github.comexternal siteGitHub1,800 starsexternal site

Reactions (1)

Similar products

by description similarity

  1. exllamav2

    A fast inference library for running LLMs locally on modern consumer-class GPUs

    github.comLibraryOpen sourcelatest reaction

    Hacker News
    No reaction
    GitHub
    4,651 stars

  2. airllm

    AirLLM 2.8T LLM inference with single 4GB GPU

    github.comOpen sourcelatest reaction

    Hacker News
    No reaction
    GitHub
    41,908 stars

  3. TensorFold

    LLM Inference Engine for Metal, CUDA and Vulkan.

    github.comOpen sourcelatest reaction

    Hacker News
    No reaction
    GitHub
    1,686 stars

  4. vllm

    A high-throughput and memory-efficient inference and serving engine for LLMs

    github.comOpen sourcelatest reaction

    Hacker News
    No reaction
    GitHub
    96,511 stars

  5. An RDMA/Infiniband Distributed Cache for Fast Inference and Training

    A high-performance RDMA distributed file system for fast LLM Inference and GPU Training.

    github.comOpen source3 reactionslatest reaction

    Hacker News
    13 pts
    GitHub
    47 stars