From Black Boxes to Conversations: Incorporating XAI in a Conversational Agent
This work addresses the need for more natural and interactive explanations of machine learning models for users, though it is an incremental step in integrating XAI with conversational agents.
The paper tackles the problem of making AI explanations conversational by incorporating Explainable AI (XAI) into a conversational agent, using an extended XAI question bank and a systematic survey of explanation methods to provide comprehensive suggestions for addressing user information needs.
The goal of Explainable AI (XAI) is to design methods to provide insights into the reasoning process of black-box models, such as deep neural networks, in order to explain them to humans. Social science research states that such explanations should be conversational, similar to human-to-human explanations. In this work, we show how to incorporate XAI in a conversational agent, using a standard design for the agent comprising natural language understanding and generation components. We build upon an XAI question bank, which we extend by quality-controlled paraphrases, to understand the user's information needs. We further systematically survey the literature for suitable explanation methods that provide the information to answer those questions, and present a comprehensive list of suggestions. Our work is the first step towards truly natural conversations about machine learning models with an explanation agent. The comprehensive list of XAI questions and the corresponding explanation methods may support other researchers in providing the necessary information to address users' demands. To facilitate future work, we release our source code and data.