Reame
a CPU inference server that gets faster as it runs. CPU-first LLM inference server on llama.cpp. Runs useful models on free-tier ARM boxes; rewriting the input made it ~6x faster and more accurate than tuning the engine. MIT, benchmarks and failures included.
셀프 호스팅오픈소스
반응 (1)
비슷한 프로덕트
설명이 비슷한 순
LLM
github.coma Rust Crate/CLI for CPU Inference of LLMs (LLaMA, GPT-NeoX, etc.)
Tiny-vLLM
github.comhigh performance LLM inference engine in C++ and CUDA
Distributed Llama
github.comRun LLMs on multiple devices in parallel
A high-performance inference engine for AI models
A tool to benchmark LLM APIs (OpenAI
llmapitest.comClaude, local/self-hosted)