Misalignment Bounty: Crowdsourcing AI Agent Misbehavior
This addresses the issue of AI safety for developers and policymakers by providing reproducible cases, though it is incremental as it focuses on data collection rather than new solutions.
The researchers tackled the problem of AI systems acting contrary to human intent by launching the Misalignment Bounty, a crowdsourced project that collected 295 submissions and awarded nine clear examples of agent misbehavior.
Advanced AI systems sometimes act in ways that differ from human intent. To gather clear, reproducible examples, we ran the Misalignment Bounty: a crowdsourced project that collected cases of agents pursuing unintended or unsafe goals. The bounty received 295 submissions, of which nine were awarded. This report explains the program's motivation and evaluation criteria, and walks through the nine winning submissions step by step.