Can an AI Agent Safely Run a Government? Existence of Probably Approximately Aligned Policies
This addresses safety concerns for deploying autonomous agents in critical applications like government, though it appears incremental as it builds on existing theory.
The paper tackles the problem of AI agent misalignment in social decision-making by providing a novel quantitative definition of alignment and proving the existence of probably approximately aligned policies, with a method to ensure verifiably safe actions.
While autonomous agents often surpass humans in their ability to handle vast and complex data, their potential misalignment (i.e., lack of transparency regarding their true objective) has thus far hindered their use in critical applications such as social decision processes. More importantly, existing alignment methods provide no formal guarantees on the safety of such models. Drawing from utility and social choice theory, we provide a novel quantitative definition of alignment in the context of social decision-making. Building on this definition, we introduce probably approximately aligned (i.e., near-optimal) policies, and we derive a sufficient condition for their existence. Lastly, recognizing the practical difficulty of satisfying this condition, we introduce the relaxed concept of safe (i.e., nondestructive) policies, and we propose a simple yet robust method to safeguard the black-box policy of any autonomous agent, ensuring all its actions are verifiably safe for the society.