Resilience of Bayesian Layer-Wise Explanations under Adversarial Attacks
This addresses the problem of unreliable AI explanations for users in security-critical applications, offering an incremental improvement by applying Bayesian methods to enhance stability.
The paper tackles the brittleness of saliency-based explanations in deterministic neural networks under adversarial attacks, even when classification remains unchanged, and demonstrates that Bayesian Neural Networks provide significantly more stable interpretations, both empirically and theoretically, with potential for more robust and interpretable assessments.
We consider the problem of the stability of saliency-based explanations of Neural Network predictions under adversarial attacks in a classification task. Saliency interpretations of deterministic Neural Networks are remarkably brittle even when the attacks fail, i.e. for attacks that do not change the classification label. We empirically show that interpretations provided by Bayesian Neural Networks are considerably more stable under adversarial perturbations of the inputs and even under direct attacks to the explanations. By leveraging recent results, we also provide a theoretical explanation of this result in terms of the geometry of the data manifold. Additionally, we discuss the stability of the interpretations of high level representations of the inputs in the internal layers of a Network. Our results demonstrate that Bayesian methods, in addition to being more robust to adversarial attacks, have the potential to provide more stable and interpretable assessments of Neural Network predictions.