Run Llama3.1 405B on a 8GB VRAM challenge [video]

비슷한 프로덕트

설명이 비슷한 순

1. Run Llama 3.1 8B in the browser

   app.wiz.chat

   HNHacker News23점

2. Llama 3.1 70B on a single RTX 3090 via NVMe-to-GPU bypassing the CPU

   github.com

   High-efficiency LLM inference engine in C++/CUDA. Run Llama 70B on RTX 3090.

   HNHacker News395점

3. L88

   news.ycombinator.com

   A Local RAG System on 8GB VRAM (Need Architecture Feedback)

   HNHacker News12점

4. Finetune Llama-3.1 2x faster in a Colab

   colab.research.google.com

   Just added Llama-3.1 support!

   HNHacker News16점

5. Can your GPU run this LLM?

   github.com

   Calculate token/s & GPU memory requirement for any LLM. Supports llama.cpp/ggml/bnb/QLoRA quantization

   HNHacker News15점