Towards a Comprehensive Human-Centred Evaluation Framework for Explainable AI
This work addresses the problem of inconsistent evaluation in XAI for researchers and practitioners, but it is incremental as it adapts an existing framework from another domain.
The authors tackled the lack of standardized human-centered evaluation procedures for explainable AI (XAI) by proposing a comprehensive framework adapted from recommender systems, aiming to standardize XAI evaluation through integrated explanation aspects and categorized metrics.
While research on explainable AI (XAI) is booming and explanation techniques have proven promising in many application domains, standardised human-centred evaluation procedures are still missing. In addition, current evaluation procedures do not assess XAI methods holistically in the sense that they do not treat explanations' effects on humans as a complex user experience. To tackle this challenge, we propose to adapt the User-Centric Evaluation Framework used in recommender systems: we integrate explanation aspects, summarise explanation properties, indicate relations between them, and categorise metrics that measure these properties. With this comprehensive evaluation framework, we hope to contribute to the human-centred standardisation of XAI evaluation.