Run Llama3.1 405B on a 8GB VRAM challenge [video]
How to run Llama3.1 405B on a 8GB VRAM.
Reactions (1)
Similar products
by description similarity
Run Llama 3.1 8B in the browser
app.wiz.chatHigh-efficiency LLM inference engine in C++/CUDA. Run Llama 70B on RTX 3090.
L88
news.ycombinator.comA Local RAG System on 8GB VRAM (Need Architecture Feedback)
Finetune Llama-3.1 2x faster in a Colab
colab.research.google.comJust added Llama-3.1 support!
Can your GPU run this LLM?
github.comCalculate token/s & GPU memory requirement for any LLM. Supports llama.cpp/ggml/bnb/QLoRA quantization