← Latest products

Running 104GB Qwen3.8-Flash-Next on 48GB Mac with at ~12 tok/s

Run a 105 GB AI model on a Mac that can't hold it. Qwen3.8-Flash-Next 4-bit, streamed from SSD on 16–64 GB Apple Silicon Macs. Fully native Swift, MLX and Metal. Offline, no Python. CLI, Swift library, and Ollama/OpenAI/Anthropic APIs. Works with Claude Code and Codex. I built slotstream, a way to run Qwen3.8-Flash-Next 4-bit on a low-memory mac starting from 16GB, a 125B parameter model that would need 100GB+ memory/RAM, thanks to expert-offloading/ssd-streaming.

CLILibraryDesktopOpen source

638 stars · GitHubexternal site

Visit github.comexternal site

Reactions (1)

Similar products

by description similarity

  1. Run an 80B Qwen in 4.3 GB of RAM on a Mac

    github.com

    and a 35B on an iPhone

    Hacker News312 pts

  2. Swift-Qwen3.8-27B

    huggingface.co

    -58.3% thinking, x1.95 speed, accuracy of xhigh

    Hacker News33 pts

  3. Run Qwen3-Next-80B on 8GB GPU at 1tok/2s throughput

    github.com

    Hacker News123 pts

  4. Qwen3.6-35B-A3B on a 16 GB M1 Pro with SSD-streamed MoE

    github.com

    Metal-first MoE inference for Apple Silicon with bounded SSD expert streaming and local OpenAI Chat, Responses, and Anthropic Messages endpoints.

    Hacker News24 pts

  5. I shrank DeepSeek V4 Flash to 57GB and it wrote a compiler on my Mac

    huggingface.co

    Text Generation model. Download, explore files and run it on Hugging Face.

    Hacker News21 pts