How good is my story? Towards quantitative metrics for evaluating LLM-generated XAI narratives
This work addresses the need for automated evaluation in XAI narratives, but it is incremental as it builds on existing methods for a specific domain.
The authors tackled the problem of evaluating LLM-generated narratives for XAI without human studies by proposing a framework with automated metrics, applying it to compare LLMs across datasets and prompts, and using it to identify challenges like hallucinations.
A rapidly developing application of LLMs in XAI is to convert quantitative explanations such as SHAP into user-friendly narratives to explain the decisions made by smaller prediction models. Evaluating the narratives without relying on human preference studies or surveys is becoming increasingly important in this field. In this work we propose a framework and explore several automated metrics to evaluate LLM-generated narratives for explanations of tabular classification tasks. We apply our approach to compare several state-of-the-art LLMs across different datasets and prompt types. As a demonstration of their utility, these metrics allow us to identify new challenges related to LLM hallucinations for XAI narratives.