Human-Centered Concept Explanations for Neural Networks
It addresses the problem of making AI models more interpretable for users by shifting from low-level feature explanations to concept-based thinking, though it is incremental as it synthesizes existing research.
The paper reviews concept-based explanations for neural networks, focusing on methods like Concept Activation Vectors to provide human-understandable insights into model predictions, with examples from synthetic and real-world applications.
Understanding complex machine learning models such as deep neural networks with explanations is crucial in various applications. Many explanations stem from the model perspective, and may not necessarily effectively communicate why the model is making its predictions at the right level of abstraction. For example, providing importance weights to individual pixels in an image can only express which parts of that particular image are important to the model, but humans may prefer an explanation which explains the prediction by concept-based thinking. In this work, we review the emerging area of concept based explanations. We start by introducing concept explanations including the class of Concept Activation Vectors (CAV) which characterize concepts using vectors in appropriate spaces of neural activations, and discuss different properties of useful concepts, and approaches to measure the usefulness of concept vectors. We then discuss approaches to automatically extract concepts, and approaches to address some of their caveats. Finally, we discuss some case studies that showcase the utility of such concept-based explanations in synthetic settings and real world applications.