ExANS
Lossless KV cache compression at 622 GB/s on H100. Once we offloaded to local disk, we realized the bottleneck is the PCIe or NIC bandwidth.
Reactions (1)
Similar products
by description similarity
We Beat MLPerf
theopenlake.comModern Storage for KV Offload and LLM Training
Plik
tycho.sytes.neta tiny FUSE filesystem with compression and deduplication
zero RAM growth
A bi-directional
hpkv.iopersisted KV store that is faster than Redis
Overfitted a 900KB Transformer to Compress a 100MB CSV into 7MB
news.ycombinator.comI built an experiment that uses an overfitted transformer and arithmetic coding to compress individual files.