Designing Incident Reporting Systems for Harms from General-Purpose AI
For policymakers and researchers designing incident reporting systems for general-purpose AI, this paper offers a structured framework and practical considerations, though it is primarily conceptual and lacks empirical validation.
The paper develops a conceptual framework for AI incident reporting systems, analyzing seven design dimensions and nine case studies from safety-critical industries to inform policy for general-purpose AI. It provides design considerations for the US context, emphasizing trade-offs in regulatory vs. non-regulatory approaches, near miss reporting, and legal clarity.
We introduce a conceptual framework and provide considerations for the institutional design of AI incident reporting systems, i.e., processes for collecting information about safety- and rights-related events caused by general-purpose AI. As general-purpose AI systems are increasingly adopted, they are causing more real-world harms and displaying the potential to cause significantly more dangerous incidents - events that did or could have caused harm to individuals, property, or the environment. Through a literature review, we develop a framework for understanding the institutional design of AI incident reporting systems, which includes seven dimensions: policy goal, actors submitting and receiving reports, type of incidents reported, level of risk materialization, enforcement of reporting, anonymity of reporters, and post-reporting actions. We then examine nine case studies of incident reporting in safety-critical industries to extract design considerations for AI incident reporting in the United States. We discuss, among other factors, differences in systems operated by regulatory vs. non-regulatory government agencies, near miss reporting, the roles of mandatory reporting thresholds and voluntary reporting channels, how to enable safety learning after reporting, sharing incident information, and clarifying legal frameworks for reporting. Our aim is to inform researchers and policymakers about when particular design choices might be more or less appropriate for AI incident reporting.