Interpretability Benchmark for Evaluating Spatial Misalignment of Prototypical Parts Explanations
This addresses interpretability issues in explainable AI for researchers and practitioners using prototypical parts networks, though it is incremental as it builds on existing models.
The paper tackles the problem of spatial misalignment in prototypical parts explanations, where prototype activation regions can be misleading due to dependencies on image areas outside those regions, and introduces a benchmark with metrics to quantify this phenomenon along with a compensation method that improves existing state-of-the-art models.
Prototypical parts-based networks are becoming increasingly popular due to their faithful self-explanations. However, their similarity maps are calculated in the penultimate network layer. Therefore, the receptive field of the prototype activation region often depends on parts of the image outside this region, which can lead to misleading interpretations. We name this undesired behavior a spatial explanation misalignment and introduce an interpretability benchmark with a set of dedicated metrics for quantifying this phenomenon. In addition, we propose a method for misalignment compensation and apply it to existing state-of-the-art models. We show the expressiveness of our benchmark and the effectiveness of the proposed compensation methodology through extensive empirical studies.