← Trending

Tangerine Index lists products launched since ChatGPT and people's reactions to them.

T-MAC

github.com

Low-bit LLM inference on CPU/NPU with lookup table

Open source

Visit github.comexternal siteGitHub1,005 starsexternal site

Reactions (1)

Similar products

by description similarity

  1. vllm

    A high-throughput and memory-efficient inference and serving engine for LLMs

    github.comOpen sourcelatest reaction

    GitHub
    96,511 stars

  2. 1Cat-vLLM

    V100 / SM70-focused vLLM engineering fork for modern LLM inference.

    github.comOpen sourcelatest reaction

    GitHub
    1,477 stars

  3. exllamav2

    A fast inference library for running LLMs locally on modern consumer-class GPUs

    github.comLibraryOpen sourcelatest reaction

    GitHub
    4,651 stars

  4. mistral.rs

    Fast, flexible LLM inference

    github.comOpen sourcelatest reaction

    GitHub
    7,817 stars

  5. TileRT

    Tile-Based Runtime for Ultra-Low-Latency LLM Inference

    github.comOpen sourcelatest reaction

    GitHub
    1,887 stars