Retrieval-Augmented Natural Language Reasoning for Explainable Visual Question Answering
This addresses the problem of resource-intensive and unstable reasoning in VQA-NLE for AI researchers, offering a more efficient solution.
The paper tackles the challenge of Visual Question Answering with Natural Language Explanation (VQA-NLE) by introducing ReRe, a retrieval-augmented model that generates accurate answers and persuasive explanations without complex networks or extra datasets, outperforming previous methods in VQA accuracy and explanation scores.
Visual Question Answering with Natural Language Explanation (VQA-NLE) task is challenging due to its high demand for reasoning-based inference. Recent VQA-NLE studies focus on enhancing model networks to amplify the model's reasoning capability but this approach is resource-consuming and unstable. In this work, we introduce a new VQA-NLE model, ReRe (Retrieval-augmented natural language Reasoning), using leverage retrieval information from the memory to aid in generating accurate answers and persuasive explanations without relying on complex networks and extra datasets. ReRe is an encoder-decoder architecture model using a pre-trained clip vision encoder and a pre-trained GPT-2 language model as a decoder. Cross-attention layers are added in the GPT-2 for processing retrieval features. ReRe outperforms previous methods in VQA accuracy and explanation score and shows improvement in NLE with more persuasive, reliability.