← Trending

Tangerine Index lists products launched since ChatGPT and people's reactions to them.

airllm

github.com

AirLLM 2.8T LLM inference with single 4GB GPU

Open source

Visit github.comexternal siteGitHub41,908 starsexternal site

Reactions (1)

Similar products

by description similarity

  1. exllamav2

    A fast inference library for running LLMs locally on modern consumer-class GPUs

    github.comLibraryOpen sourcelatest reaction

    GitHub
    4,651 stars

  2. exllamav3

    An optimized quantization and inference library for running LLMs locally on modern consumer-class GPUs

    github.comLibraryOpen sourcelatest reaction

    GitHub
    1,800 stars

  3. vllm

    A high-throughput and memory-efficient inference and serving engine for LLMs

    github.comOpen sourcelatest reaction

    GitHub
    96,511 stars

  4. T-MAC

    Low-bit LLM inference on CPU/NPU with lookup table

    github.comOpen sourcelatest reaction

    GitHub
    1,005 stars

  5. 1Cat-vLLM

    V100 / SM70-focused vLLM engineering fork for modern LLM inference.

    github.comOpen sourcelatest reaction

    GitHub
    1,477 stars