Unpredictability of AI
This addresses a foundational challenge in AI safety by identifying a key limitation for researchers and developers.
The paper tackles the problem of predicting actions of smarter-than-human AI systems, proving it is impossible to precisely and consistently predict these actions even with knowledge of terminal goals.
The young field of AI Safety is still in the process of identifying its challenges and limitations. In this paper, we formally describe one such impossibility result, namely Unpredictability of AI. We prove that it is impossible to precisely and consistently predict what specific actions a smarter-than-human intelligent system will take to achieve its objectives, even if we know terminal goals of the system. In conclusion, impact of Unpredictability on AI Safety is discussed.