Learning to Explain with Complemental Examples
This work addresses the need for interpretable AI systems by providing complementary explanations, though it is incremental in building on existing multimodal explanation methods.
The paper tackles the problem of generating multimodal explanations for classification decisions by producing both linguistic explanations and complementary visual examples, using a framework that maximizes interaction information to ensure complementarity, and demonstrates effectiveness across several datasets.
This paper addresses the generation of explanations with visual examples. Given an input sample, we build a system that not only classifies it to a specific category, but also outputs linguistic explanations and a set of visual examples that render the decision interpretable. Focusing especially on the complementarity of the multimodal information, i.e., linguistic and visual examples, we attempt to achieve it by maximizing the interaction information, which provides a natural definition of complementarity from an information theoretical viewpoint. We propose a novel framework to generate complemental explanations, on which the joint distribution of the variables to explain, and those to be explained is parameterized by three different neural networks: predictor, linguistic explainer, and example selector. Explanation models are trained collaboratively to maximize the interaction information to ensure the generated explanation are complemental to each other for the target. The results of experiments conducted on several datasets demonstrate the effectiveness of the proposed method.