← Trending

Tangerine Index lists products launched since ChatGPT and people's reactions to them.

candle-vllm

github.com

Efficent platform for inference and serving local LLMs including an OpenAI compatible API server.

Open source

Visit github.comexternal siteGitHub739 starsexternal site

Reactions (1)

Similar products

by description similarity

  1. vllm

    A high-throughput and memory-efficient inference and serving engine for LLMs

    GitHub96,543 starsgithub.comOpen sourcelatest reaction

  2. guidellm

    Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs

    GitHub1,762 starsgithub.comOpen sourcelatest reaction

  3. 1Cat-vLLM

    V100 / SM70-focused vLLM engineering fork for modern LLM inference.

    GitHub1,479 starsgithub.comOpen sourcelatest reaction

  4. catwalk

    🐈 A collection of LLM inference providers and models

    GitHub822 starsgithub.comOpen sourcelatest reaction

  5. rtp-llm

    RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.

    GitHub1,394 starsgithub.comOpen sourcelatest reaction