LLM Thematic Generalization Benchmark
Thematic Generalization Benchmark: measures how effectively various LLMs can infer a narrow or specific "theme" (category/rule) from a small set of examples and anti-examples, then detect which item truly fits that theme among a collection of misleading candidates.
오픈소스
반응 (1)
비슷한 프로덕트
설명이 비슷한 순
Benchmark evaluating LLMs on their ability to create and resist disinformation. Includes comprehensive testing across major models (Claude, GPT-4, Gemini, Llama, etc.) with standardized evaluation metrics.
New LLM outperforming GPT-3.5
news.ycombinator.comRefuel LLM (84.2%) outperforms trained human annotators (80.4%), GPT-3-5-turbo (81.3%), PaLM-2 (82.3%) and Claude (79.3%) across a benchmark of 15 text labeling datasets.
An LLM purpose built for data annotation
news.ycombinator.comoutperforms GPT-3.5
A multi-source LLM benchmark across text, image, and audio that measures JSON value accuracy per field, not just schema compliance. 7 metrics, full leaderboard.
LLM Creative Story‑Writing Benchmark V3
github.comThis benchmark tests how well LLMs incorporate a set of 10 mandatory story elements (characters, objects, core concepts, attributes, motivations, etc.) in a short creative story