Artificial General Intelligence, Existential Risk, and Human Risk Perception
This addresses the problem of understanding public and expert risk perceptions for AGI, which is crucial for policy and safety discussions, though it is incremental as it builds on existing risk analysis without proposing new solutions.
The paper examines perceptions of existential risk from artificial general intelligence (AGI), finding that both experts and non-experts view AGI as a greater and more rapidly increasing threat compared to other risks like nuclear war or climate change, with agreement on its pressing nature but unclear underlying reasons.
Artificial general intelligence (AGI) does not yet exist, but given the pace of technological development in artificial intelligence, it is projected to reach human-level intelligence within roughly the next two decades. After that, many experts expect it to far surpass human intelligence and to do so rapidly. The prospect of superintelligent AGI poses an existential risk to humans because there is no reliable method for ensuring that AGI goals stay aligned with human goals. Drawing on publicly available forecaster and opinion data, the author examines how experts and non-experts perceive risk from AGI. The findings indicate that the perceived risk of a world catastrophe or extinction from AGI is greater than for other existential risks. The increase in perceived risk over the last year is also steeper for AGI than for other existential threats (e.g., nuclear war or human-caused climate change). That AGI is a pressing existential risk is something on which experts and non-experts agree, but the basis for such agreement currently remains obscure.