Cuts Long Horizon Inference Costs by 50% via external KV Cache Offload
OpenLake is a high performance storage engine for efficient LLM inference and GPU Training. We built OpenLake because KV caches are outgrowing GPU memory.
Open source
Reactions (1)
Similar products
by description similarity
A high-performance RDMA distributed file system for fast LLM Inference and GPU Training.
Free Inference Engineer and Model Training Roadmap
inferquest.orgFree, gamified roadmaps for LLM engineering: an Inference Engineering path (KV caches, CUDA kernels, production vLLM serving) and a Model Training path (pretraining on a budget, scaling laws, SFT/DPO/GRPO) — 185 tasks with auto-verified milestones instead of a paper certificate.
Butter
docs.butter.deva muscle memory cache for LLMs
Tiny-vLLM
github.comhigh performance LLM inference engine in C++ and CUDA
OpenGraviton
opengraviton.github.ioRun 500B+ parameter models on a consumer Mac Mini