Preventing Repeated Real World AI Failures by Cataloging Incidents: The AI Incident Database
This addresses the lack of collective memory for AI failures, helping companies and society prevent recurring harms.
The paper tackles the problem of repeated real-world AI failures by creating the AI Incident Database, a collection of over 1,000 incident reports to enable avoidance and mitigation through research and development.
Mature industrial sectors (e.g., aviation) collect their real world failures in incident databases to inform safety improvements. Intelligent systems currently cause real world harms without a collective memory of their failings. As a result, companies repeatedly make the same mistakes in the design, development, and deployment of intelligent systems. A collection of intelligent system failures experienced in the real world (i.e., incidents) is needed to ensure intelligent systems benefit people and society. The AI Incident Database is an incident collection initiated by an industrial/non-profit cooperative to enable AI incident avoidance and mitigation. The database supports a variety of research and development use cases with faceted and full text search on more than 1,000 incident reports archived to date.