← Trending

Tangerine Index lists products launched since ChatGPT and people's reactions to them.

optillm

github.com

Optimizing inference proxy for LLMs

Open source

Visit github.comexternal siteGitHub4,338 starsexternal site

Reactions (1)

Similar products

by description similarity

  1. vllm

    A high-throughput and memory-efficient inference and serving engine for LLMs

    github.comOpen sourcelatest reaction

    GitHub
    96,511 stars

  2. rtp-llm

    RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.

    github.comOpen sourcelatest reaction

    GitHub
    1,394 stars

  3. sonar

    Large-scale LLM inference engine

    github.comOpen sourcelatest reaction

    GitHub
    1,905 stars

  4. web-llm

    High-performance In-browser LLM Inference Engine

    github.comOpen sourcelatest reaction

    GitHub
    19,361 stars

  5. mistral.rs

    Fast, flexible LLM inference

    github.comOpen sourcelatest reaction

    GitHub
    7,817 stars