AI Research Considerations for Human Existential Safety (ARCHES)
It addresses the problem of long-term human survival from AI risks for researchers and policymakers, but is incremental as it builds on existing risk analysis frameworks.
The report examines how AI research can be directed to address existential risks to humanity's survival over the next century, introducing the concept of 'prepotence' to delineate these risks and analyzing contemporary research directions for their potential benefits and side effects to existential safety.
Framed in positive terms, this report examines how technical AI research might be steered in a manner that is more attentive to humanity's long-term prospects for survival as a species. In negative terms, we ask what existential risks humanity might face from AI development in the next century, and by what principles contemporary technical research might be directed to address those risks. A key property of hypothetical AI technologies is introduced, called \emph{prepotence}, which is useful for delineating a variety of potential existential risks from artificial intelligence, even as AI paradigms might shift. A set of \auxref{dirtot} contemporary research \directions are then examined for their potential benefit to existential safety. Each research direction is explained with a scenario-driven motivation, and examples of existing work from which to build. The research directions present their own risks and benefits to society that could occur at various scales of impact, and in particular are not guaranteed to benefit existential safety if major developments in them are deployed without adequate forethought and oversight. As such, each direction is accompanied by a consideration of potentially negative side effects.