← Latest products

New eval from SWE-bench team evalutes LMs based on goals not tickets

CodeClash: Benchmarking Goal-Oriented Software Engineering. Current evals test LMs on tasks: "fix this bug," "write a test" But we code to achieve goals: maximize revenue, cut costs, win users Meet CodeClash: LMs compete via their codebases across multi-round tournaments to achieve high-level goals.

Visit codeclash.aiexternal site

Reactions (1)

Similar products

by description similarity

  1. New Benchmark from SWE-bench team is 0% solved

    programbench.com

    ProgramBench evaluates whether language models can rebuild programs from scratch.

    Hacker News24 pts

  2. Cheddar-bench

    github.com

    unsupervised benchmark for coding agents

    Hacker News9 pts

  3. Leaping

    news.ycombinator.com

    Open-source debugging with LLMs

    Hacker News15 pts

  4. A new benchmark for testing LLMs for deterministic outputs

    interfaze.ai

    A multi-source LLM benchmark across text, image, and audio that measures JSON value accuracy per field, not just schema compliance. 7 metrics, full leaderboard.

    Hacker News60 pts

  5. Stage

    stagereview.app

    Putting humans back in control of code review

    Hacker News130 pts