Tangerine Index lists products launched since ChatGPT and people's reactions to them.
vLLM-2080Ti-Definitive
github.com
The definitive vLLM runtime for dual RTX 2080 Ti 22GB + NVLink, delivering Qwen 27B local inference with maximum 200+ tok/s single-request decode with support of FP8 weight ( Join Discord :https://discord.gg/VFqVVySdMS )
Open source
Reactions (1)
Similar products
by description similarity
MemStitch
Zero-copy context bridging for vLLM (25x TTFT speedup)
GPTQModel
LLM model quantization (compression) toolkit with HW acceleration support for Nvidia (CUDA), AMD (ROCm), Intel GPU (XPU), Huawei Ascent (NPU), and Intel/AMD/Apple CPU via HF, vLLM, and SGLang.
airllm
AirLLM 2.8T LLM inference with single 4GB GPU
OS Megakernel that match M5 Max Tok/w at 2x the Throughput on RTX 3090
LLM speculative inference server for heterogeneous hardware & consumer GPUs
Llama 3.1 70B on a single RTX 3090 via NVMe-to-GPU bypassing the CPU
High-efficiency LLM inference engine in C++/CUDA. Run Llama 70B on RTX 3090.
