Running 104GB Qwen3.8-Flash-Next on 48GB Mac with at ~12 tok/s
Run a 105 GB AI model on a Mac that can't hold it. Qwen3.8-Flash-Next 4-bit, streamed from SSD on 16–64 GB Apple Silicon Macs. Fully native Swift, MLX and Metal. Offline, no Python. CLI, Swift library, and Ollama/OpenAI/Anthropic APIs. Works with Claude Code and Codex. I built slotstream, a way to run Qwen3.8-Flash-Next 4-bit on a low-memory mac starting from 16GB, a 125B parameter model that would need 100GB+ memory/RAM, thanks to expert-offloading/ssd-streaming.
CLI라이브러리데스크톱오픈소스
반응 (1)
비슷한 프로덕트
설명이 비슷한 순
and a 35B on an iPhone
Swift-Qwen3.8-27B
huggingface.co-58.3% thinking, x1.95 speed, accuracy of xhigh
Metal-first MoE inference for Apple Silicon with bounded SSD expert streaming and local OpenAI Chat, Responses, and Anthropic Messages endpoints.
Text Generation model. Download, explore files and run it on Hugging Face.