← Trending

Tangerine Index lists products launched since ChatGPT and people's reactions to them.

LLMSpeculativeSampling

github.com

Fast inference from large lauguage models via speculative decoding

Open source

Visit github.comexternal siteGitHub926 starsexternal site

Reactions (1)

Similar products

by description similarity

  1. speculators

    A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM

    GitHub946 starsgithub.comLibraryOpen sourcelatest reaction

  2. chitu

    High-performance inference framework for large language models, focusing on efficiency, flexibility, and availability.

    GitHub2,999 starsgithub.comLibraryOpen sourcelatest reaction

  3. LLMs following grammar for general-purpose PLs

    Efficient and general syntactical decoding for Large Language Models

    Hacker News11 ptsGitHub341 starsgithub.comOpen source2 reactionslatest reaction

  4. SpecForge

    Train speculative decoding models effortlessly and port them smoothly to SGLang serving.

    GitHub1,274 starsgithub.comOpen sourcelatest reaction

  5. mistral.rs

    Fast, flexible LLM inference

    Hacker News5 ptsGitHub7,820 starsgithub.comOpen source3 reactionslatest reaction