A Taxonomy for Design and Evaluation of Prompt-Based Natural Language Explanations
This work addresses the problem of improving AI governance and transparency for researchers, auditors, and policymakers, but it is incremental as it adapts existing XAI literature to a new context.
The authors tackled the need for structured approaches to evaluate prompt-based natural language explanations in AI by developing a taxonomy across context, generation, and evaluation dimensions, providing a framework for stakeholders to enhance transparency.
Effective AI governance requires structured approaches for stakeholders to access and verify AI system behavior. With the rise of large language models, Natural Language Explanations (NLEs) are now key to articulating model behavior, which necessitates a focused examination of their characteristics and governance implications. We draw on Explainable AI (XAI) literature to create an updated XAI taxonomy, adapted to prompt-based NLEs, across three dimensions: (1) Context, including task, data, audience, and goals; (2) Generation and Presentation, covering generation methods, inputs, interactivity, outputs, and forms; and (3) Evaluation, focusing on content, presentation, and user-centered properties, as well as the setting of the evaluation. This taxonomy provides a framework for researchers, auditors, and policymakers to characterize, design, and enhance NLEs for transparent AI systems.