Llm.sql
Run a 640MB LLM on SQLite, with 210MB peak RSS and 7.4 tok/s. The motivation: Edge LLMs are getting better, but hardware remains a bottleneck, especially RAM (size and bandwidth).
Reactions (1)
Similar products
by description similarity
Rekal
github.comLong-term memory for LLMs in a single SQLite file
ClickHouse / Postgres
Tsellm
github.comInteractive SQLite shell with LLM support
Improve Text-to-SQL Accuracy with LLM
github.comA repository that contains models, datasets, and fine-tuning techniques for DB-GPT, with the purpose of enhancing model performance in Text-to-SQL
Local persistent memory store for LLM applications including claude desktop, github copilot, codex, antigravity, etc.