← 최신 프로덕트

Cuts Long Horizon Inference Costs by 50% via external KV Cache Offload

OpenLake is a high performance storage engine for efficient LLM inference and GPU Training. We built OpenLake because KV caches are outgrowing GPU memory.

오픈소스

2,633 스타 · GitHub외부 사이트

github.com 방문외부 사이트

반응 (1)

비슷한 프로덕트

설명이 비슷한 순

  1. An RDMA/Infiniband Distributed Cache for Fast Inference and Training

    github.com

    A high-performance RDMA distributed file system for fast LLM Inference and GPU Training.

    Hacker News13점

  2. Free Inference Engineer and Model Training Roadmap

    inferquest.org

    Free, gamified roadmaps for LLM engineering: an Inference Engineering path (KV caches, CUDA kernels, production vLLM serving) and a Model Training path (pretraining on a budget, scaling laws, SFT/DPO/GRPO) — 185 tasks with auto-verified milestones instead of a paper certificate.

    Hacker News16점

  3. Butter

    docs.butter.dev

    a muscle memory cache for LLMs

    Hacker News23점

  4. Tiny-vLLM

    github.com

    high performance LLM inference engine in C++ and CUDA

    Hacker News205점

  5. OpenGraviton

    opengraviton.github.io

    Run 500B+ parameter models on a consumer Mac Mini

    Hacker News13점