Blog comments
nice looking, open source – Talkyard. Uses a bit much memory (2 GB min, 4 better), should be possible to get below 0.5 GB if pre-compiling JVM stuff.
반응 (1)
비슷한 프로덕트
설명이 비슷한 순
28 tok/s on 8GB GPU
Go ~18%
L88
news.ycombinator.comA Local RAG System on 8GB VRAM (Need Architecture Feedback)
I built a memory-safe web server in Rust
ferronweb.orgThe web server that I am building is currently in beta, so any feedback is welcome.
We made glhf.chat
glhf.chatrun almost any open-source LLM, including 405B