AI Agents Should be Regulated Based on the Extent of Their Autonomous Operations
This addresses regulatory gaps for AI safety, especially for high-risk autonomous agents, but is incremental as it builds on existing discussions about existential risks and regulation.
The paper argues that AI agents should be regulated based on their level of autonomy, particularly focusing on long-term planning capabilities that pose risks of human extinction and global catastrophes, and proposes using action sequences as a better measure than current metrics like computational scale.
This position paper argues that AI agents should be regulated by the extent to which they operate autonomously. AI agents with long-term planning and strategic capabilities can pose significant risks of human extinction and irreversible global catastrophes. While existing regulations often focus on computational scale as a proxy for potential harm, we argue that such measures are insufficient for assessing the risks posed by agents whose capabilities arise primarily from inference-time computation. To support our position, we discuss relevant regulations and recommendations from scientists regarding existential risks, as well as the advantages of using action sequences -- which reflect the degree of an agent's autonomy -- as a more suitable measure of potential impact than existing metrics that rely on observing environmental states.