Misalignment or misuse? The AGI alignment tradeoff
This addresses the critical safety problem for AI developers and policymakers by highlighting a tradeoff between alignment and misuse risks, with incremental insights into technical and social factors.
The paper tackles the dual risks of misaligned AGI causing catastrophic failures and aligned AGI enabling catastrophic misuse by humans, arguing that some alignment approaches can avoid increasing misuse risk, though many current techniques likely do.
Creating systems that are aligned with our goals is seen as a leading approach to create safe and beneficial AI in both leading AI companies and the academic field of AI safety. We defend the view that misaligned AGI - future, generally intelligent (robotic) AI agents - poses catastrophic risks. At the same time, we support the view that aligned AGI creates a substantial risk of catastrophic misuse by humans. While both risks are severe and stand in tension with one another, we show that - in principle - there is room for alignment approaches which do not increase misuse risk. We then investigate how the tradeoff between misalignment and misuse looks empirically for different technical approaches to AI alignment. Here, we argue that many current alignment techniques and foreseeable improvements thereof plausibly increase risks of catastrophic misuse. Since the impacts of AI depend on the social context, we close by discussing important social factors and suggest that to reduce the risk of a misuse catastrophe due to aligned AGI, techniques such as robustness, AI control methods and especially good governance seem essential.