A taxonomic system for failure cause analysis of open source AI incidents
This work addresses the lack of mandatory incident reporting in AI safety by providing a method for inferring causes from open-source incidents, though it is incremental as it builds on existing databases and expert analysis.
The paper tackles the problem of analyzing causes of AI incidents by developing a taxonomic system for classifying technical failure factors using expert knowledge and the AI Incident Database, resulting in a structured ontology and workflow for annotating incidents.
While certain industrial sectors (e.g., aviation) have a long history of mandatory incident reporting complete with analytical findings, the practice of artificial intelligence (AI) safety benefits from no such mandate and thus analyses must be performed on publicly known ``open source'' AI incidents. Although the exact causes of AI incidents are seldom known by outsiders, this work demonstrates how to apply expert knowledge on the population of incidents in the AI Incident Database (AIID) to infer the potential and likely technical causative factors that contribute to reported failures and harms. We present early work on a taxonomic system that covers a cascade of interrelated incident factors, from system goals (nearly always known) to methods / technologies (knowable in many cases) and technical failure causes (subject to expert analysis) of the implicated systems. We pair this ontology structure with a comprehensive classification workflow that leverages expert knowledge and community feedback, resulting in taxonomic annotations grounded by incident data and human expertise.