← Latest products

I run 30B 22tok/s

109tok/s not novel,6GB/16GB RAM overcoming llama.cpp. Run a 110B on a 2016 PC with 16 GB RAM. Know your tok/s before you download. Placement beats budget: predicts speed + memory fit for any GGUF on your exact hardware, self-calibrates, emits the exact llama.cpp command — or 'quantprobe auto' does it all. Falsification-tested laws; misses published at full size. pip install quantprobe. Democratisation of local AI is key.

Open source

92 stars · GitHubexternal site

Visit github.comexternal site

Reactions (1)

Similar products

by description similarity

  1. Shoehorn

    notactuallytreyanastasio.github.io

    Quantize any model down to run on your machine

    Hacker News99 pts

  2. Can your GPU run this LLM?

    github.com

    Calculate token/s & GPU memory requirement for any LLM. Supports llama.cpp/ggml/bnb/QLoRA quantization

    Hacker News15 pts

  3. Llama.cpp Tutorial 2026

    news.ycombinator.com

    Run GGUF Models Locally on CPU and GPU

    Hacker News13 pts

  4. Llama.go

    github.com

    port of llama.cpp to pure Go

    Hacker News25 pts

  5. Llama 3.1 70B on a single RTX 3090 via NVMe-to-GPU bypassing the CPU

    github.com

    High-efficiency LLM inference engine in C++/CUDA. Run Llama 70B on RTX 3090.

    Hacker News395 pts