Learning to Explain: A Model-Agnostic Framework for Explaining Black Box Models
This addresses the need for interpretable AI in vision tasks, offering a flexible solution for various model architectures, though it is incremental as it builds on existing post-hoc explanation techniques.
The paper tackles the problem of explaining black box vision models by introducing Learning to Explain (LTX), a model-agnostic framework that generates explanation maps, and it shows that LTX significantly outperforms state-of-the-art methods in explainability metrics.
We present Learning to Explain (LTX), a model-agnostic framework designed for providing post-hoc explanations for vision models. The LTX framework introduces an "explainer" model that generates explanation maps, highlighting the crucial regions that justify the predictions made by the model being explained. To train the explainer, we employ a two-stage process consisting of initial pretraining followed by per-instance finetuning. During both stages of training, we utilize a unique configuration where we compare the explained model's prediction for a masked input with its original prediction for the unmasked input. This approach enables the use of a novel counterfactual objective, which aims to anticipate the model's output using masked versions of the input image. Importantly, the LTX framework is not restricted to a specific model architecture and can provide explanations for both Transformer-based and convolutional models. Through our evaluations, we demonstrate that LTX significantly outperforms the current state-of-the-art in explainability across various metrics.