Identifying Important Group of Pixels using Interactions
This work addresses the interpretability of image classifiers for researchers and practitioners, but it is incremental as it builds on existing game-theoretic concepts with computational improvements.
The authors tackled the problem of visualizing pixel contributions in image classifiers by proposing MoXI, a method that identifies groups of pixels with high prediction confidence using Shapley values and interactions, achieving better performance than Grad-CAM, Attention rollout, and Shapley value in experiments.
To better understand the behavior of image classifiers, it is useful to visualize the contribution of individual pixels to the model prediction. In this study, we propose a method, MoXI ($\textbf{Mo}$del e$\textbf{X}$planation by $\textbf{I}$nteractions), that efficiently and accurately identifies a group of pixels with high prediction confidence. The proposed method employs game-theoretic concepts, Shapley values and interactions, taking into account the effects of individual pixels and the cooperative influence of pixels on model confidence. Theoretical analysis and experiments demonstrate that our method better identifies the pixels that are highly contributing to the model outputs than widely-used visualization by Grad-CAM, Attention rollout, and Shapley value. While prior studies have suffered from the exponential computational cost in the computation of Shapley value and interactions, we show that this can be reduced to quadratic cost for our task. The code is available at https://github.com/KosukeSumiyasu/MoXI.