Tangerine Index는 ChatGPT 이후 나온 프로덕트와 사람들의 반응을 싣습니다.
LLGTRT
github.com
TensorRT-LLM+Rust server w/ OpenAI-compat and Structured Output
오픈소스
제작자가 쓴 설명
TensorRT-LLM server with Structured Outputs (JSON) built with Rust. I wanted to share a project I've been working on for the past few weeks: llgtrt.
반응 (2)
비슷한 프로덕트
설명이 비슷한 순
TensorRT-LLM
TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.
smg
Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM, TokenSpeed, SGLang, OpenAI, Gemini & more. Industry-first gRPC pipeline, KV cache-aware routing, chat history, tokenization caching, Responses API, embeddings, WASM plugins, MCP, and multi-tenant auth.
Offeryn
Build Tools for LLMs in Rust, with Model Context Protocol
LLM
a Rust Crate/CLI for CPU Inference of LLMs (LLaMA, GPT-NeoX, etc.)
llrt
LLRT (Low Latency Runtime) is an experimental, lightweight JavaScript runtime designed to address the growing demand for fast and efficient Serverless applications.
