Towards credible visual model interpretation with path attribution
This work improves the credibility of model interpretation tools for deep learning practitioners, though it is incremental as it builds on existing path attribution frameworks.
The paper addresses reliability issues in path attribution methods for interpreting deep visual models, identifying conditions that avoid counter-intuitive results and preserve axiomatic properties, and demonstrates consistent performance gains over baselines across multiple datasets and metrics.
Originally inspired by game-theory, path attribution framework stands out among the post-hoc model interpretation tools due to its axiomatic nature. However, recent developments show that this framework can still suffer from counter-intuitive results. Moreover, specifically for deep visual models, the existing path-based methods also fall short on conforming to the original intuitions that are the basis of the claimed axiomatic properties of this framework. We address these problems with a systematic investigation, and pinpoint the conditions in which the counter-intuitive results can be avoided for deep visual model interpretation with the path attribution strategy. We also devise a scheme to preclude the conditions in which visual model interpretation can invalidate the axiomatic properties of path attribution. These insights are combined into a method that enables reliable visual model interpretation. Our findings are establish empirically with multiple datasets, models and evaluation metrics. Extensive experiments show a consistent performance gain of our method over the baselines.