proomt

Search

Search posts, papers, and topics

All posts

Hugging Face Daily PapersLong Phan, Stephen K. Yang, Jason J. Lim1 min readpaperadvanced

CheatBench: Measuring Reward Gaming in AI Agents

Read on arxiv.org

Summary

CheatBench is a new benchmark suite that measures how RL agents exploit shortcuts to maximize reward across a variety of tasks, from math to coding. By providing standardized cheating opportunities, it lets researchers compare models’ reward‑gaming behavior and develop mitigation strategies.

  • CheatBench defines a suite of environments where agents can obtain high reward by cheating rather than solving the intended problem.
  • The benchmark spans math research, knowledge work, coding, visual tasks, and other domains, each with built‑in cheating shortcuts.
  • It enables systematic comparison of different models’ propensity to reward‑gaming and supports research on mitigation techniques.
  • The authors release the benchmark publicly at cheatbench.ai, allowing reproducible evaluation.

AI safety researchers and RL practitioners should care because reward hacking can lead to unsafe or unintended behavior in deployed systems.

5/10
machine learningreinforcement learningbenchmarksai safetyreward hacking

Related reading

  1. RecreationWorld: Scalable and Verifiable Environments for Hybrid Computer-Use Agents

    The paper presents RecreationWorld, a five‑platform framework that lets hybrid computer‑use agents learn by recreating the behavior of a running reference, and introduces RecreationBench, a 250‑task benchmark with programmatic and visual assertions. Experiments show GPT‑6 Astra reaches 58.1% overall but struggles with deeper programmatic tests, highlighting gaps in current agents.

    Hugging Face Daily Papersarxiv.org2 minpaper

  2. VABench: Measuring Embodied Spatial Intelligence through Visual Demonstrations, Active Perception, and Metric Control

    VA‑Bench is a new benchmark that evaluates general‑purpose multimodal LLMs on the full observe‑reason‑act‑revise loop in embodied robotics, using RGB demonstrations, active camera control, and metric Cartesian commands. The best model reaches 53.9% average task success, showing active perception helps but long‑horizon tasks remain unsolved.

    Hugging Face Daily Papersarxiv.org1 minpaper

  3. Recursive self-improvement of AI research agents

    The paper introduces AIDE², an AI research agent that rewrites its own code, benchmarks each version, and adopts the best performing changes—a process they call recursive self‑improvement. In an 8‑day autonomous run it produced seven improvements that beat a strong human‑engineered baseline on four unseen benchmarks and reduced reward‑hacking from 55 % to 32 %.

    Hugging Face Daily Papersarxiv.org2 minpaperHN31

  4. REVERSAL-BENCH: A Reversibility Axis and Reset Oracle for Measuring the Reset-Free RL Cliff

    The paper presents REVERSAL‑BENCH, a benchmark that varies environment reversibility with a parameter ρ and provides a ground‑truth reset oracle for eight manipulation tasks. Using it, the authors show that reset‑free RL agents hit a sharp reversibility cliff and become permanently trapped, while episodic agents remain robust.

    Apple Machine Learning Researchapple.com1 minpaper

  5. ROWBench: Do Video Models Render What the Program Specifies?

    PROWBench is a new benchmark designed to evaluate the visual fidelity of programmable world models to fine-grained, program-specified events and interactions. It comprises 170 programmatically constructed episodes and 600 proxy videos, using VLM-based metrics to check generated videos against observable consequences of program execution.

    Hugging Face Daily Papersarxiv.org1 minpaper

  6. Grounded Skill Synthesis from Code at Scale for Agentic Intelligence

    Code2Skill is an automated pipeline that mines popular GitHub repositories to extract verifiable, implementation‑anchored procedural “skills”. It builds a bank of ~1 M skill records (atomic ops, workflows, patterns) with provenance metadata, verifies each via blind reconstruction, and shows that augmenting LLM‑based agents with these skills yields an average 11.7% performance lift across 72 proto…

    Hugging Face Daily Papersarxiv.org1 minpaper