← 트렌딩

Tangerine Index는 ChatGPT 이후 나온 프로덕트와 사람들의 반응을 싣습니다.

vLLM-2080Ti-Definitive

github.com

The definitive vLLM runtime for dual RTX 2080 Ti 22GB + NVLink, delivering Qwen 27B local inference with maximum 200+ tok/s single-request decode with support of FP8 weight ( Join Discord :https://discord.gg/VFqVVySdMS )

오픈소스

github.com 방문외부 사이트GitHub1,213 스타외부 사이트

반응 (1)

비슷한 프로덕트

설명이 비슷한 순

  1. MemStitch

    Zero-copy context bridging for vLLM (25x TTFT speedup)

    Hacker News12점GitHub스타 22github.com오픈소스반응 2개최근 반응

  2. GPTQModel

    LLM model quantization (compression) toolkit with HW acceleration support for Nvidia (CUDA), AMD (ROCm), Intel GPU (XPU), Huawei Ascent (NPU), and Intel/AMD/Apple CPU via HF, vLLM, and SGLang.

    GitHub스타 1,289github.com오픈소스최근 반응

  3. airllm

    AirLLM 2.8T LLM inference with single 4GB GPU

    Hacker News232점GitHub스타 42,081github.com오픈소스반응 2개최근 반응

  4. OS Megakernel that match M5 Max Tok/w at 2x the Throughput on RTX 3090

    LLM speculative inference server for heterogeneous hardware & consumer GPUs

    Hacker News6점GitHub스타 2,991github.com오픈소스반응 2개최근 반응

  5. Llama 3.1 70B on a single RTX 3090 via NVMe-to-GPU bypassing the CPU

    High-efficiency LLM inference engine in C++/CUDA. Run Llama 70B on RTX 3090.

    Hacker News395점GitHub스타 469github.com오픈소스반응 2개최근 반응