Run Qwen3-Next-80B on 8GB GPU at 1tok/2s throughput

Similar products

by description similarity

1. Running 104GB Qwen3.8-Flash-Next on 48GB Mac with at ~12 tok/s

   github.com

   Run a 105 GB AI model on a Mac that can't hold it. Qwen3.8-Flash-Next 4-bit, streamed from SSD on 16–64 GB Apple Silicon Macs. Fully native Swift, MLX and Metal. Offline, no Python. CLI, Swift library, and Ollama/OpenAI/Anthropic APIs. Works with Claude Code and Codex.

   HNHacker News240 pts

2. Fine-tune an 8B model on a 4 GB laptop GPU

   github.com

   Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU.

   HNHacker News139 pts

3. Can your GPU run this LLM?

   github.com

   Calculate token/s & GPU memory requirement for any LLM. Supports llama.cpp/ggml/bnb/QLoRA quantization

   HNHacker News15 pts

4. wc-GPU

   github.com

   The Unix util `wc` running on a GPU

   HNHacker News15 pts

5. Local fine tuning for Mistral and SDXL

   docs.helix.ml

   GPU mem/latency optimization

   HNHacker News36 pts