On the Connection Between Adversarial Robustness and Saliency Map Interpretability
This work addresses the problem of improving model interpretability for researchers and practitioners in machine learning, but it is incremental as it builds on prior observations to quantify and test the connection.
The paper investigates the relationship between adversarial robustness and saliency map interpretability in neural networks, finding that models trained to be more robust produce more interpretable saliency maps, with theoretical confirmation for linear models and experimental validation for non-linear cases.
Recent studies on the adversarial vulnerability of neural networks have shown that models trained to be more robust to adversarial attacks exhibit more interpretable saliency maps than their non-robust counterparts. We aim to quantify this behavior by considering the alignment between input image and saliency map. We hypothesize that as the distance to the decision boundary grows,so does the alignment. This connection is strictly true in the case of linear models. We confirm these theoretical findings with experiments based on models trained with a local Lipschitz regularization and identify where the non-linear nature of neural networks weakens the relation.