Running PrismML's Bonsai inside DRAM by breaking DDR4 timing rules
The excitement surrounding PrismML’s 1-bit/ternary Bonsai models has the industry closely watching how smartphone giants, particularly Apple, will implement LLMs on edge devices.
Reactions (1)
Similar products
by description similarity
Bonsai 1.7B ternary model at 442T/s on M4 Max
agents2agents.aiA drop-in optimized inference build of Bonsai 1.7B for Apple Silicon. ~42% faster decode (tg128) and ~9% faster prefill (pp512) on M-series GPUs. All Metal kernels written and tuned by ata, our autonomous engineering agent.
Bonsai
github.comA Competitive Ternary Weight LLM
1-Bit Bonsai
prismml.comthe First Commercially Viable 1-Bit LLMs
80% faster
github.com50% less memory, 0% loss of accuracy Llama finetuning
Data Bonsai
github.coma Python package to clean your data with LLMs