Enhancing Model Interpretability with Local Attribution over Global Exploration
This work addresses the challenge of generating reliable explanations for AI model decisions, which is crucial for trust and debugging in applications like healthcare or finance, though it appears incremental as it builds on existing attribution methods.
The paper tackles the problem of model interpretability by addressing how intermediate states in attribution methods can reach out-of-distribution spaces and distort feature importance, proposing a Local Attribution algorithm that improves attribution effectiveness by an average of 38.21% compared to state-of-the-art methods.
In the field of artificial intelligence, AI models are frequently described as `black boxes' due to the obscurity of their internal mechanisms. It has ignited research interest on model interpretability, especially in attribution methods that offers precise explanations of model decisions. Current attribution algorithms typically evaluate the importance of each parameter by exploring the sample space. A large number of intermediate states are introduced during the exploration process, which may reach the model's Out-of-Distribution (OOD) space. Such intermediate states will impact the attribution results, making it challenging to grasp the relative importance of features. In this paper, we firstly define the local space and its relevant properties, and we propose the Local Attribution (LA) algorithm that leverages these properties. The LA algorithm comprises both targeted and untargeted exploration phases, which are designed to effectively generate intermediate states for attribution that thoroughly encompass the local space. Compared to the state-of-the-art attribution methods, our approach achieves an average improvement of 38.21\% in attribution effectiveness. Extensive ablation studies in our experiments also validate the significance of each component in our algorithm. Our code is available at: https://github.com/LMBTough/LA/