Pencil Puzzle Bench
LLM Benchmark for Multi-Step Verifiable Reasoning. I've been working on applying LLMs to long-context, verifiable problems over the past year, and today I'm releasing a benchmark of 62,000 pencil puzzles across 94 types (sudoku, nonori, slitherlink, etc.).
Reactions (1)
Similar products
by description similarity
A multi-source LLM benchmark across text, image, and audio that measures JSON value accuracy per field, not just schema compliance. 7 metrics, full leaderboard.
Benchmark evaluating LLMs on their ability to create and resist disinformation. Includes comprehensive testing across major models (Claude, GPT-4, Gemini, Llama, etc.) with standardized evaluation metrics.
LLM Divergent Thinking Creativity Benchmark. LLMs generate 25 unique words that start with a given letter with no connections to each other or to 50 initial random words.
LocalScore
localscore.aiLocal LLM Benchmark
ML, Biotech, Hardware, and Coordination Problems. Sometimes I write about hard problems and how to solve them.