BashArena: A Control Setting for Highly Privileged AI Agents
This addresses security risks from misaligned AI agents in system administration, offering a benchmark for AI control research, though it is incremental as it builds on existing control frameworks.
The paper tackles the problem of AI agents with elevated privileges potentially causing damage if misaligned, by introducing BashArena, a setting with 637 Linux tasks and sabotage objectives for studying control techniques. It finds that Claude Sonnet 4.5 can execute sabotage undetected by GPT-4.1 mini 26% of the time at 4% FPR, providing a baseline for control protocols.
Future AI agents might run autonomously with elevated privileges. If these agents are misaligned, they might abuse these privileges to cause serious damage. The field of AI control develops techniques that make it harder for misaligned AIs to cause such damage, while preserving their usefulness. We introduce BashArena, a setting for studying AI control techniques in security-critical environments. BashArena contains 637 Linux system administration and infrastructure engineering tasks in complex, realistic environments, along with four sabotage objectives (execute malware, exfiltrate secrets, escalate privileges, and disable firewall) for a red team to target. We evaluate multiple frontier LLMs on their ability to complete tasks, perform sabotage undetected, and detect sabotage attempts. Claude Sonnet 4.5 successfully executes sabotage while evading monitoring by GPT-4.1 mini 26% of the time, at 4% trajectory-wise FPR. Our findings provide a baseline for designing more effective control protocols in BashArena. We release the dataset as a ControlArena setting and share our task generation pipeline.