Running PrismML's Bonsai inside DRAM by breaking DDR4 timing rules
The excitement surrounding PrismML’s 1-bit/ternary Bonsai models has the industry closely watching how smartphone giants, particularly Apple, will implement LLMs on edge devices.
반응 (1)
비슷한 프로덕트
설명이 비슷한 순
Bonsai 1.7B ternary model at 442T/s on M4 Max
agents2agents.aiA drop-in optimized inference build of Bonsai 1.7B for Apple Silicon. ~42% faster decode (tg128) and ~9% faster prefill (pp512) on M-series GPUs. All Metal kernels written and tuned by ata, our autonomous engineering agent.
Bonsai
github.comA Competitive Ternary Weight LLM
1-Bit Bonsai
prismml.comthe First Commercially Viable 1-Bit LLMs
80% faster
github.com50% less memory, 0% loss of accuracy Llama finetuning
Data Bonsai
github.coma Python package to clean your data with LLMs