Show Why the Answer is Correct! Towards Explainable AI using Compositional Temporal Attention
This addresses the need for reliable, explainable AI in safety-critical applications like autonomous systems and cybersecurity, though it is incremental as it builds on existing VQA methods.
The paper tackles the lack of explainability and poor handling of complex questions in Visual Question Answering (VQA) models by proposing a Dynamic Neural Network with compositional temporal attention, which outperforms previous approaches on VQA2.0 and CLEVR datasets and provides reasoning for predictions.
Visual Question Answering (VQA) models have achieved significant success in recent times. Despite the success of VQA models, they are mostly black-box models providing no reasoning about the predicted answer, thus raising questions for their applicability in safety-critical such as autonomous systems and cyber-security. Current state of the art fail to better complex questions and thus are unable to exploit compositionality. To minimize the black-box effect of these models and also to make them better exploit compositionality, we propose a Dynamic Neural Network (DMN), which can understand a particular question and then dynamically assemble various relatively shallow deep learning modules from a pool of modules to form a network. We incorporate compositional temporal attention to these deep learning based modules to increase compositionality exploitation. This results in achieving better understanding of complex questions and also provides reasoning as to why the module predicts a particular answer. Experimental analysis on the two benchmark datasets, VQA2.0 and CLEVR, depicts that our model outperforms the previous approaches for Visual Question Answering task as well as provides better reasoning, thus making it reliable for mission critical applications like safety and security.