OpenGraviton
Run 500B+ parameter models on a consumer Mac Mini. The system combines several techniques to drastically reduce memory and compute requirements: • 1.58-bit ternary quantization ({-1, 0, +1}) for ~10x compression • dynamic sparsity with Top-K pruning and MoE routing • mmap-based layer streaming to load weights directly from NVMe SSDs • speculative de.
Reactions (1)
Similar products
by description similarity
By combining 1.58-bit ternary quantization, dynamic sparsity with Top-K pruning and MoE routing, and mmap-based layer streaming, OpenGraviton can run models far larger than your system RAM—even on a Mac Mini.
OpenLake is a high performance storage engine for efficient LLM inference and GPU Training
Metal-first MoE inference for Apple Silicon with bounded SSD expert streaming and local OpenAI Chat, Responses, and Anthropic Messages endpoints.
80% faster
github.com50% less memory, 0% loss of accuracy Llama finetuning
DPO, GRPO)