I nerfed our coding agents on purpose
Tl;dr: I trained a classifier to route to the least expensive model and reasoning depth to complete the request.
Reactions (1)
Similar products
by description similarity
I, like many of you, have really struggled with benchmarking and comparing models, from missing code, to opaque experiment parameters slowing the process.
I Built a Debugging Challenge for the AI Coding Age
theincidentchallenge.comIt's hard to evaluate and improve your skills, when AI is writing the code.
it might help you guys too
Chestnut
chestnut.soThe antidote to AI-induced skill atrophy