Tangerine Index lists products launched since ChatGPT and people's reactions to them.
LLGTRT
github.com
TensorRT-LLM+Rust server w/ OpenAI-compat and Structured Output
Open source
In the makers' words
TensorRT-LLM server with Structured Outputs (JSON) built with Rust. I wanted to share a project I've been working on for the past few weeks: llgtrt.
Reactions (2)
Similar products
by description similarity
TensorRT-LLM
TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.
smg
Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM, TokenSpeed, SGLang, OpenAI, Gemini & more. Industry-first gRPC pipeline, KV cache-aware routing, chat history, tokenization caching, Responses API, embeddings, WASM plugins, MCP, and multi-tenant auth.
Offeryn
Build Tools for LLMs in Rust, with Model Context Protocol
LLM
a Rust Crate/CLI for CPU Inference of LLMs (LLaMA, GPT-NeoX, etc.)
llrt
LLRT (Low Latency Runtime) is an experimental, lightweight JavaScript runtime designed to address the growing demand for fast and efficient Serverless applications.
