May I Ask a Follow-up Question? Understanding the Benefits of Conversations in Neural Network Explainability
This addresses the need for more interactive and user-customized explanations in XAI, though it is incremental as it builds on existing static methods.
The paper tackled the problem of static explanations in explainable AI (XAI) by investigating if free-form conversations enhance user comprehension and trust, finding that conversations significantly improved comprehension, acceptance, trust, and collaboration.
Research in explainable AI (XAI) aims to provide insights into the decision-making process of opaque AI models. To date, most XAI methods offer one-off and static explanations, which cannot cater to the diverse backgrounds and understanding levels of users. With this paper, we investigate if free-form conversations can enhance users' comprehension of static explanations, improve acceptance and trust in the explanation methods, and facilitate human-AI collaboration. Participants are presented with static explanations, followed by a conversation with a human expert regarding the explanations. We measure the effect of the conversation on participants' ability to choose, from three machine learning models, the most accurate one based on explanations and their self-reported comprehension, acceptance, and trust. Empirical results show that conversations significantly improve comprehension, acceptance, trust, and collaboration. Our findings highlight the importance of customized model explanations in the format of free-form conversations and provide insights for the future design of conversational explanations.