How I topped the HuggingFace open LLM leaderboard on two gaming GPUs
ML, Biotech, Hardware, and Coordination Problems. Sometimes I write about hard problems and how to solve them. I found that duplicating a specific block of 7 middle layers in Qwen2-72B, without modifying any weights, improved performance across all Open LLM Leaderboard benchmarks and took #1.
반응 (1)
비슷한 프로덕트
설명이 비슷한 순
80% faster
github.com50% less memory, 0% loss of accuracy Llama finetuning
Can your GPU run this LLM?
github.comCalculate token/s & GPU memory requirement for any LLM. Supports llama.cpp/ggml/bnb/QLoRA quantization
Duplicate 3 layers in a 24B LLM
github.comlogical deduction .22→.76. No training
LocalScore
localscore.aiLocal LLM Benchmark
This project collects GPU benchmarks from various cloud providers and compares them to fixed per token costs. Use our tool for efficient LLM GPU selections and cost-effective AI models. LLM provider price comparison, gpu benchmarks to price per token calculation, gpu benchmark table