Towards AI Forensics: Did the Artificial Intelligence System Do It?
It addresses forensic accountability for AI systems, particularly those potentially malicious by design, which is an incremental step in AI safety and regulation.
The paper tackles the problem of determining whether an AI system caused a specific harmful event and what triggered it, proposing a conceptual framework and strategies for forensic investigation, with evaluation on convolutional neural networks illustrating challenges and ideas for identifying malicious AI.
Artificial intelligence (AI) makes decisions impacting our daily lives in an increasingly autonomous manner. Their actions might cause accidents, harm, or, more generally, violate regulations. Determining whether an AI caused a specific event and, if so, what triggered the AI's action, are key forensic questions. We provide a conceptualization of the problems and strategies for forensic investigation. We focus on AI that is potentially ``malicious by design'' and grey box analysis. Our evaluation using convolutional neural networks illustrates challenges and ideas for identifying malicious AI.