NanoRL
RL training for LLMs in ~1,800 lines. One RL training loop that runs CartPole on a laptop and disaggregated async RLVR on a GPU cluster. About 1,800 lines across 7 files. No Ray, no TRL, no DeepSpeed. It's small enough to read in an afternoon, and it's meant to be forked, not imported. The smallest async RL trainer I could write: one loop that runs REINFORCE on CartPole on a laptop and async GRPO on a cluster (e.g.
Open source
Reactions (1)
Similar products
by description similarity
DPO, GRPO)
A ~9M parameter LLM that talks like a small fish.
Tiny-vLLM
github.comhigh performance LLM inference engine in C++ and CUDA
Terminal-Bench-RL
github.comTraining long-horizon terminal agents with RL
LLM App
github.combuild a realtime LLM app in 30 lines, with no vector database