← Latest products

Shoehorn

Quantize any model down to run on your machine. Quantize a BF16 GGUF with an imatrix to exactly fit your VRAM, then run it with llama.cpp. Every spare megabyte spent where it buys the most quality. Working on Mac, Linux, and Windows now.

Open source

93 stars · GitHubexternal site

Visit notactuallytreyanastasio.github.ioexternal site

Reactions (2)

Similar products

by description similarity

  1. I run 30B 22tok/s

    github.com

    109tok/s not novel,6GB/16GB RAM overcoming llama.cpp

    Hacker News5 pts

  2. Can your GPU run this LLM?

    github.com

    Calculate token/s & GPU memory requirement for any LLM. Supports llama.cpp/ggml/bnb/QLoRA quantization

    Hacker News15 pts

  3. Llama.cpp Tutorial 2026

    news.ycombinator.com

    Run GGUF Models Locally on CPU and GPU

    Hacker News13 pts

  4. Quant Picker

    vettedconsumer.com

    which GGUF file fits your model and machine

    Hacker News20 pts

  5. Llama 3.1 70B on a single RTX 3090 via NVMe-to-GPU bypassing the CPU

    github.com

    High-efficiency LLM inference engine in C++/CUDA. Run Llama 70B on RTX 3090.

    Hacker News395 pts