Robust Explanations for Visual Question Answering
This work addresses the need for more reliable and coherent explanations in VQA systems, which is incremental as it builds on existing methods to enhance correlation and robustness.
The paper tackles the problem of generating robust visual and textual explanations for visual question answering that correlate well with answers, addressing challenges like poor correlation and misaligned visual focus. The proposed method improves textual and visual justification, as evaluated on the VQA-X dataset, and demonstrates robustness against noise-based attacks.
In this paper, we propose a method to obtain robust explanations for visual question answering(VQA) that correlate well with the answers. Our model explains the answers obtained through a VQA model by providing visual and textual explanations. The main challenges that we address are i) Answers and textual explanations obtained by current methods are not well correlated and ii) Current methods for visual explanation do not focus on the right location for explaining the answer. We address both these challenges by using a collaborative correlated module which ensures that even if we do not train for noise based attacks, the enhanced correlation ensures that the right explanation and answer can be generated. We further show that this also aids in improving the generated visual and textual explanations. The use of the correlated module can be thought of as a robust method to verify if the answer and explanations are coherent. We evaluate this model using VQA-X dataset. We observe that the proposed method yields better textual and visual justification that supports the decision. We showcase the robustness of the model against a noise-based perturbation attack using corresponding visual and textual explanations. A detailed empirical analysis is shown. Here we provide source code link for our model \url{https://github.com/DelTA-Lab-IITK/CCM-WACV}.