Unifying VXAI: A Systematic Review and Framework for the Evaluation of Explainable AI
This work addresses the problem of inconsistent evaluation in XAI for researchers and practitioners, though it is incremental as it synthesizes existing literature rather than proposing new methods.
The authors tackled the lack of standardized evaluation protocols in Explainable AI (XAI) by conducting a systematic review of 362 publications and introducing a unified framework (VXAI) that categorizes 41 metric groups and a three-dimensional scheme to improve comparability and selection of evaluation metrics.
Modern AI systems frequently rely on opaque black-box models, most notably Deep Neural Networks, whose performance stems from complex architectures with millions of learned parameters. While powerful, their complexity poses a major challenge to trustworthiness, particularly due to a lack of transparency. Explainable AI (XAI) addresses this issue by providing human-understandable explanations of model behavior. However, to ensure their usefulness and trustworthiness, such explanations must be rigorously evaluated. Despite the growing number of XAI methods, the field lacks standardized evaluation protocols and consensus on appropriate metrics. To address this gap, we conduct a systematic literature review following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines and introduce a unified framework for the eValuation of XAI (VXAI). We identify 362 relevant publications and aggregate their contributions into 41 functionally similar metric groups. In addition, we propose a three-dimensional categorization scheme spanning explanation type, evaluation contextuality, and explanation quality desiderata. Our framework provides the most comprehensive and structured overview of VXAI to date. It supports systematic metric selection, promotes comparability across methods, and offers a flexible foundation for future extensions.