Duplicate 3 layers in a 24B LLM
logical deduction .22→.76. No training. I replicated Ng's RYS method and found that duplicating 3 specific layers in Qwen2.5-32B boosts reasoning by 17% and duplicating layers 12-14 in Devstral-24B improves logical deduction from 0.22→0.76 on BBH — no training, no weight changes, just routing hidden states through the same circuit twice. Tools included. Two AMD GPUs, one evening. I replicated David Ng's RYS method ( https://dnhkng.github.io/posts/rys/ ) on consumer AMD GPUs (RX 7900 XT + RX 6950 XT) and found something I didn't expect.
Open source
Reactions (1)
Similar products
by description similarity
ML, Biotech, Hardware, and Coordination Problems. Sometimes I write about hard problems and how to solve them.
DPO, GRPO)
Cuq
github.comFormal Verification of Rust GPU Kernels
Speeding up LLM inference 2x times (possibly)
asciinema.orgHere's a project I've been working on for the last few months.
Can your GPU run this LLM?
github.comCalculate token/s & GPU memory requirement for any LLM. Supports llama.cpp/ggml/bnb/QLoRA quantization