← 최신 프로덕트

KTransformers:671B DeepSeek-R1 on a Single Machine-286 tokens/s Prefill

A Flexible Framework for Experiencing Heterogeneous LLM Inference/Fine-tune Optimizations. We are excited to share the new version of KTransformers, a flexible framework designed for cutting-edge LLM inference optimizations!

라이브러리오픈소스

19,647 스타 · GitHub외부 사이트

github.com 방문외부 사이트

반응 (2)

비슷한 프로덕트

설명이 비슷한 순

  1. Tiny-vLLM

    github.com

    high performance LLM inference engine in C++ and CUDA

    Hacker News205점

  2. Free Inference Engineer and Model Training Roadmap

    inferquest.org

    Free, gamified roadmaps for LLM engineering: an Inference Engineering path (KV caches, CUDA kernels, production vLLM serving) and a Model Training path (pretraining on a budget, scaling laws, SFT/DPO/GRPO) — 185 tasks with auto-verified milestones instead of a paper certificate.

    Hacker News16점

  3. Cuts Long Horizon Inference Costs by 50% via external KV Cache Offload

    github.com

    OpenLake is a high performance storage engine for efficient LLM inference and GPU Training

    Hacker News22점

  4. Speeding up LLM inference 2x times (possibly)

    asciinema.org

    Here's a project I've been working on for the last few months.

    Hacker News419점

  5. 80% faster

    github.com

    50% less memory, 0% loss of accuracy Llama finetuning

    Hacker News385점