When Can You Trust Your Explanations? A Robustness Analysis on Feature Importances
This work addresses the need for standardized validation criteria in XAI to ensure trustworthy AI systems, providing a framework for practitioners, though it is incremental in focusing on robustness analysis for feature importances.
The paper tackles the problem of evaluating the trustworthiness of explanations in Explainable AI by analyzing their robustness to non-adversarial perturbations, proposing a novel approach based on the manifold hypothesis and an ensemble method to aggregate explanations, with experimental results on neural networks applied to tabular datasets highlighting the importance of robust explanations.
Recent legislative regulations have underlined the need for accountable and transparent artificial intelligence systems and have contributed to a growing interest in the Explainable Artificial Intelligence (XAI) field. Nonetheless, the lack of standardized criteria to validate explanation methodologies remains a major obstacle to developing trustworthy systems. We address a crucial yet often overlooked aspect of XAI, the robustness of explanations, which plays a central role in ensuring trust in both the system and the provided explanation. To this end, we propose a novel approach to analyse the robustness of neural network explanations to non-adversarial perturbations, leveraging the manifold hypothesis to produce new perturbed datapoints that resemble the observed data distribution. We additionally present an ensemble method to aggregate various explanations, showing how merging explanations can be beneficial for both understanding the model's decision and evaluating the robustness. The aim of our work is to provide practitioners with a framework for evaluating the trustworthiness of model explanations. Experimental results on feature importances derived from neural networks applied to tabular datasets highlight the importance of robust explanations in practical applications.