← Trending

Tangerine Index lists products launched since ChatGPT and people's reactions to them.

llm-d

github.com

State of the art inference performance with modern accelerators on Kubernetes

Open source

Visit github.comexternal siteGitHub5,166 starsexternal site

Reactions (1)

Similar products

by description similarity

  1. vllm

    A high-throughput and memory-efficient inference and serving engine for LLMs

    github.comOpen sourcelatest reaction

    GitHub
    96,511 stars

  2. tokenspeed

    TokenSpeed is a speed-of-light LLM inference engine.

    github.comOpen sourcelatest reaction

    GitHub
    2,885 stars

  3. exllamav2

    A fast inference library for running LLMs locally on modern consumer-class GPUs

    github.comLibraryOpen sourcelatest reaction

    GitHub
    4,651 stars

  4. T-MAC

    Low-bit LLM inference on CPU/NPU with lookup table

    github.comOpen sourcelatest reaction

    GitHub
    1,005 stars

  5. TensorFold

    LLM Inference Engine for Metal, CUDA and Vulkan.

    github.comOpen sourcelatest reaction

    GitHub
    1,686 stars