Understanding and Avoiding AI Failures: A Practical Guide
This work addresses the practical challenge of preventing AI failures for developers and policymakers, but it is incremental as it builds on existing theories without introducing new methods.
The paper tackles the problem of increasing AI accidents by developing a framework that combines normal accident theory, high reliability theory, open systems theory, and AI safety principles to understand and quantify risks, focusing on system properties near accidents to guide safety efforts.
As AI technologies increase in capability and ubiquity, AI accidents are becoming more common. Based on normal accident theory, high reliability theory, and open systems theory, we create a framework for understanding the risks associated with AI applications. In addition, we also use AI safety principles to quantify the unique risks of increased intelligence and human-like qualities in AI. Together, these two fields give a more complete picture of the risks of contemporary AI. By focusing on system properties near accidents instead of seeking a root cause of accidents, we identify where attention should be paid to safety for current generation AI systems.