Games for AI Control: Models of Safety Evaluations of AI Deployment Protocols
This work addresses safety evaluation for AI deployment, particularly for untrusted language models as programming assistants, but is incremental as it formalizes existing red-teaming concepts.
The paper tackles the problem of evaluating safety protocols for deploying untrusted AI systems by introducing AI-Control Games, a formal model of red-teaming exercises as multi-objective, partially observable, stochastic games, and demonstrates its utility by improving over empirical studies and evaluating protocols in new settings.
To evaluate the safety and usefulness of deployment protocols for untrusted AIs, AI Control uses a red-teaming exercise played between a protocol designer and an adversary. This paper introduces AI-Control Games, a formal decision-making model of the red-teaming exercise as a multi-objective, partially observable, stochastic game. We also introduce methods for finding optimal protocols in AI-Control Games, by reducing them to a set of zero-sum partially observable stochastic games. We apply our formalism to model, evaluate and synthesise protocols for deploying untrusted language models as programming assistants, focusing on Trusted Monitoring protocols, which use weaker language models and limited human assistance. Finally, we demonstrate the utility of our formalism by showcasing improvements over empirical studies in existing settings, evaluating protocols in new settings, and analysing how modelling assumptions affect the safety and usefulness of protocols.