KVSplit
Run 2-3x longer contexts on Apple Silicon. Run larger LLMs with longer contexts on Apple Silicon by using differentiated precision for KV cache quantization. KVSplit enables 8-bit keys & 4-bit values, reducing memory by 59% with <1% quality loss. Includes benchmarking, visualization, and one-command setup. Optimized for M1/M2/M3 Macs with Metal support. I discovered that in LLM inference, keys and values in the KV cache have very different quantization sensitivities.
Open source
Reactions (1)
Similar products
by description similarity
By combining 1.58-bit ternary quantization, dynamic sparsity with Top-K pruning and MoE routing, and mmap-based layer streaming, OpenGraviton can run models far larger than your system RAM—even on a Mac Mini.
OpenGraviton
opengraviton.github.ioRun 500B+ parameter models on a consumer Mac Mini
OpenLake is a high performance storage engine for efficient LLM inference and GPU Training
80% faster
github.com50% less memory, 0% loss of accuracy Llama finetuning
MemStitch
github.comZero-copy context bridging for vLLM (25x TTFT speedup)