Impact of Adversarial Attacks on Deep Learning Model Explainability
This addresses the problem of ensuring trustworthy AI by evaluating the robustness of explainability techniques for practitioners, but it is incremental as it focuses on existing methods and metrics.
The paper investigates how adversarial attacks affect the explainability of deep learning models, finding that while model accuracy drops significantly (e.g., from 89.94% to 58.73% under FGSM attacks), explanation metrics like IoU and RMSE show negligible changes, indicating these metrics may be insensitive to such perturbations.
In this paper, we investigate the impact of adversarial attacks on the explainability of deep learning models, which are commonly criticized for their black-box nature despite their capacity for autonomous feature extraction. This black-box nature can affect the perceived trustworthiness of these models. To address this, explainability techniques such as GradCAM, SmoothGrad, and LIME have been developed to clarify model decision-making processes. Our research focuses on the robustness of these explanations when models are subjected to adversarial attacks, specifically those involving subtle image perturbations that are imperceptible to humans but can significantly mislead models. For this, we utilize attack methods like the Fast Gradient Sign Method (FGSM) and the Basic Iterative Method (BIM) and observe their effects on model accuracy and explanations. The results reveal a substantial decline in model accuracy, with accuracies dropping from 89.94% to 58.73% and 45.50% under FGSM and BIM attacks, respectively. Despite these declines in accuracy, the explanation of the models measured by metrics such as Intersection over Union (IoU) and Root Mean Square Error (RMSE) shows negligible changes, suggesting that these metrics may not be sensitive enough to detect the presence of adversarial perturbations.