AI Systems of Concern
This addresses safety concerns for society from future AI, but is incremental as it builds on existing frameworks without new empirical results.
The paper tackles the problem of potential dangers from advanced AI systems with 'Property X' characteristics like agent-like behavior and strategic awareness, arguing that these are intrinsically dangerous and hard to control when combined with high capabilities, and proposes indicators and governance interventions to limit such systems.
Concerns around future dangers from advanced AI often centre on systems hypothesised to have intrinsic characteristics such as agent-like behaviour, strategic awareness, and long-range planning. We label this cluster of characteristics as "Property X". Most present AI systems are low in "Property X"; however, in the absence of deliberate steering, current research directions may rapidly lead to the emergence of highly capable AI systems that are also high in "Property X". We argue that "Property X" characteristics are intrinsically dangerous, and when combined with greater capabilities will result in AI systems for which safety and control is difficult to guarantee. Drawing on several scholars' alternative frameworks for possible AI research trajectories, we argue that most of the proposed benefits of advanced AI can be obtained by systems designed to minimise this property. We then propose indicators and governance interventions to identify and limit the development of systems with risky "Property X" characteristics.