Uncovering Bugs in Formal Explainers: A Case Study with PyXAI
This addresses the reliability of formal XAI tools for users in AI safety and interpretability, though it is incremental as it focuses on validation rather than new explainability methods.
The paper tackled the problem of validating practical implementations of formal explainable AI (XAI) methods, developing a novel methodology and finding that the PyXAI explainer produced incorrect explanations on most datasets tested.
Formal explainable artificial intelligence (XAI) offers unique theoretical guarantees of rigor when compared to other non-formal methods of explainability. However, little attention has been given to the validation of practical implementations of formal explainers. This paper develops a novel methodology for validating formal explainers and reports on the assessment of the publicly available formal explainer PyXAI. The paper documents the existence of incorrect explanations computed by PyXAI on most of the datasets analyzed in the experiments, thereby confirming the importance of the proposed novel methodology for the validation of formal explainers.