Exploring Effectiveness of Explanations for Appropriate Trust: Lessons from Cognitive Psychology
This work addresses the challenge of human-AI collaboration by providing a framework for explanation designers, though it is incremental as it builds on existing cognitive psychology insights without introducing new algorithms.
The paper tackles the problem of designing effective AI explanations to foster appropriate human trust and understanding, drawing from cognitive psychology to propose four key components (perception, semantics, intent, and user & context) for improving explanation design, with an example application in food calorie estimation.
The rapid development of Artificial Intelligence (AI) requires developers and designers of AI systems to focus on the collaboration between humans and machines. AI explanations of system behavior and reasoning are vital for effective collaboration by fostering appropriate trust, ensuring understanding, and addressing issues of fairness and bias. However, various contextual and subjective factors can influence an AI system explanation's effectiveness. This work draws inspiration from findings in cognitive psychology to understand how effective explanations can be designed. We identify four components to which explanation designers can pay special attention: perception, semantics, intent, and user & context. We illustrate the use of these four explanation components with an example of estimating food calories by combining text with visuals, probabilities with exemplars, and intent communication with both user and context in mind. We propose that the significant challenge for effective AI explanations is an additional step between explanation generation using algorithms not producing interpretable explanations and explanation communication. We believe this extra step will benefit from carefully considering the four explanation components outlined in our work, which can positively affect the explanation's effectiveness.