← Latest products

RewardHackBench

Using sandboxes to stop agents from cheating. Benchmarking execution environments ability to prevent reward hacking in agent evals. ever since the cheating agents ( https://debugml.github.io/cheating-agents/ ) paper came out, revealing reward hacking was 4x more prevalent than previously estimated, i've been looking into how we can deal with the issue the common approach (taken by the tbench team) is post hoc trajectory analysis.

Open source

18 stars · GitHubexternal site

Visit github.comexternal site

Reactions (1)

Similar products

by description similarity

  1. Letting Claude automate fleets of browser sandboxes

    twitter.com

    Agents like OpenClaw and CC are super good at using browsers but, similar to code sandboxes, they should be able to control these browser in the background, in parallel, without getting blocked by captchas.

    Hacker News6 pts

  2. A registry of agent benchmarks (including many OSS agent trajectories)

    explorer.invariantlabs.ai

    A better way of testing, inspecting, and analyzing AI Agent traces. - invariantlabs-ai/explorer

    Hacker News6 pts

  3. Cheddar-bench

    github.com

    unsupervised benchmark for coding agents

    Hacker News9 pts

  4. Open-Source Gateway to Stop Tool-Abusing Prompt Injections

    archestra.ai

    If you're building LLM agents that use tools, you're probably worried about prompt injection attacks that can hijack those tools.

    Hacker News9 pts

  5. A searchable public database of cheaters in video games

    gamingherd.com

    Watch this video if you're interested into the different scales of cheats that can be used, pretty much undetectable: https://www.youtube.com/watch?v=ox1mBxrVulk&ab_channel=HaiX Currently, Valve does not seem to have a viable solution for dealing with cheaters nor are they transparent about their in

    Hacker News29 pts