Sensible AI: Re-imagining Interpretability and Explainability using Sensemaking Theory
This work addresses the challenge of designing interpretable AI for stakeholders by providing a human-centered framework, though it builds incrementally on existing advocacy for stakeholder needs.
The paper tackles the problem of making AI explanations more effective by shifting focus from improving the explanation artifact to considering the human recipient, proposing a framework based on sensemaking theory that incorporates properties like identity and social context to shape understanding.
Understanding how ML models work is a prerequisite for responsibly designing, deploying, and using ML-based systems. With interpretability approaches, ML can now offer explanations for its outputs to aid human understanding. Though these approaches rely on guidelines for how humans explain things to each other, they ultimately solve for improving the artifact -- an explanation. In this paper, we propose an alternate framework for interpretability grounded in Weick's sensemaking theory, which focuses on who the explanation is intended for. Recent work has advocated for the importance of understanding stakeholders' needs -- we build on this by providing concrete properties (e.g., identity, social context, environmental cues, etc.) that shape human understanding. We use an application of sensemaking in organizations as a template for discussing design guidelines for Sensible AI, AI that factors in the nuances of human cognition when trying to explain itself.