← 최신 프로덕트

I run 30B 22tok/s

109tok/s not novel,6GB/16GB RAM overcoming llama.cpp. Run a 110B on a 2016 PC with 16 GB RAM. Know your tok/s before you download. Placement beats budget: predicts speed + memory fit for any GGUF on your exact hardware, self-calibrates, emits the exact llama.cpp command — or 'quantprobe auto' does it all. Falsification-tested laws; misses published at full size. pip install quantprobe. Democratisation of local AI is key.

오픈소스

92 스타 · GitHub외부 사이트

github.com 방문외부 사이트

반응 (1)

비슷한 프로덕트

설명이 비슷한 순

  1. Shoehorn

    notactuallytreyanastasio.github.io

    Quantize any model down to run on your machine

    Hacker News99점

  2. Can your GPU run this LLM?

    github.com

    Calculate token/s & GPU memory requirement for any LLM. Supports llama.cpp/ggml/bnb/QLoRA quantization

    Hacker News15점

  3. Llama.cpp Tutorial 2026

    news.ycombinator.com

    Run GGUF Models Locally on CPU and GPU

    Hacker News13점

  4. Llama.go

    github.com

    port of llama.cpp to pure Go

    Hacker News25점

  5. Llama 3.1 70B on a single RTX 3090 via NVMe-to-GPU bypassing the CPU

    github.com

    High-efficiency LLM inference engine in C++/CUDA. Run Llama 70B on RTX 3090.

    Hacker News395점