ExANS
Lossless KV cache compression at 622 GB/s on H100. Once we offloaded to local disk, we realized the bottleneck is the PCIe or NIC bandwidth.
반응 (1)
비슷한 프로덕트
설명이 비슷한 순
We Beat MLPerf
theopenlake.comModern Storage for KV Offload and LLM Training
Plik
tycho.sytes.neta tiny FUSE filesystem with compression and deduplication
zero RAM growth
A bi-directional
hpkv.iopersisted KV store that is faster than Redis
Overfitted a 900KB Transformer to Compress a 100MB CSV into 7MB
news.ycombinator.comI built an experiment that uses an overfitted transformer and arithmetic coding to compress individual files.