Evaluation of Black-Box XAI Approaches for Predictors of Values of Boolean Formulae
This work addresses the problem of objectively evaluating XAI methods for AI researchers and practitioners, but it is incremental as it builds on existing tools and focuses on a specific domain.
The paper tackled the challenge of evaluating explainable AI (XAI) approaches for tabular data predicting Boolean functions by proposing a formal measure based on actual causality and introducing a new tool B-ReX, which achieved a Jensen-Shannon divergence of 0.072 ± 0.012 on a benchmark.
Evaluating explainable AI (XAI) approaches is a challenging task in general, due to the subjectivity of explanations. In this paper, we focus on tabular data and the specific use case of AI models predicting the values of Boolean functions. We extend the previous work in this domain by proposing a formal and precise measure of importance of variables based on actual causality, and we evaluate state-of-the-art XAI tools against this measure. We also present a novel XAI tool B-ReX, based on the existing tool ReX, and demonstrate that it is superior to other black-box XAI tools on a large-scale benchmark. Specifically, B-ReX achieves a Jensen-Shannon divergence of 0.072 $\pm$ 0.012 on random 10-valued Boolean formulae