Instance-wise or Class-wise? A Tale of Neighbor Shapley for Concept-based Explanation
This tackles the problem of model transparency for users in high-stakes domains, but appears incremental as it builds on existing concept-based explanation approaches.
The paper addresses the lack of interpretability in deep neural networks, which is critical for applications like crime judgment and medical diagnosis, by investigating methods to interpret model knowledge.
Deep neural networks have demonstrated remarkable performance in many data-driven and prediction-oriented applications, and sometimes even perform better than humans. However, their most significant drawback is the lack of interpretability, which makes them less attractive in many real-world applications. When relating to the moral problem or the environmental factors that are uncertain such as crime judgment, financial analysis, and medical diagnosis, it is essential to mine the evidence for the model's prediction (interpret model knowledge) to convince humans. Thus, investigating how to interpret model knowledge is of paramount importance for both academic research and real applications.