← Latest products

Running Gemma-4 26B at 124 tokens/SEC on a CPU

no GPU. Gemma-4 26B at 124 tokens/sec on a desktop CPU, no GPU. The byte budget, the quantization recipe, the roofline, and the failed attempts, measured. I wanted to know how fast a 26B mixture-of-experts model could run on a desktop CPU with no GPU.

Visit apeg.devexternal site

Reactions (1)

Similar products

by description similarity

  1. Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series Mac

    github.com

    Gemma 4 26B-A4B inference in ~2 GB of RAM on any M-series MacBook

    Hacker News919 pts

  2. Gemma Gem

    github.com

    AI model embedded in a browser – no API keys, no cloud

    Hacker News156 pts

  3. Can your GPU run this LLM?

    github.com

    Calculate token/s & GPU memory requirement for any LLM. Supports llama.cpp/ggml/bnb/QLoRA quantization

    Hacker News15 pts

  4. Horizon

    github.com

    GPU-accelerated infinite-canvas terminal in Rust

    Hacker News84 pts

  5. Mcptoon

    github.com

    Token-efficient MCP CLI client

    Hacker News73 pts