Reame
a CPU inference server that gets faster as it runs. CPU-first LLM inference server on llama.cpp. Runs useful models on free-tier ARM boxes; rewriting the input made it ~6x faster and more accurate than tuning the engine. MIT, benchmarks and failures included.
Self-hostedOpen source
Reactions (1)
Similar products
by description similarity
LLM
github.coma Rust Crate/CLI for CPU Inference of LLMs (LLaMA, GPT-NeoX, etc.)
Tiny-vLLM
github.comhigh performance LLM inference engine in C++ and CUDA
Distributed Llama
github.comRun LLMs on multiple devices in parallel
A high-performance inference engine for AI models
A tool to benchmark LLM APIs (OpenAI
llmapitest.comClaude, local/self-hosted)