High-Order Attention Models for Visual Question Answering
This work addresses the challenge of multimodal integration for cognitive-like tasks, offering a novel approach that enhances performance in visual question answering.
The paper tackles the problem of learning high-order correlations between visual and textual modalities to improve attention mechanisms in visual question answering, achieving state-of-the-art performance on the standard VQA dataset.
The quest for algorithms that enable cognitive abilities is an important part of machine learning. A common trait in many recently investigated cognitive-like tasks is that they take into account different data modalities, such as visual and textual input. In this paper we propose a novel and generally applicable form of attention mechanism that learns high-order correlations between various data modalities. We show that high-order correlations effectively direct the appropriate attention to the relevant elements in the different data modalities that are required to solve the joint task. We demonstrate the effectiveness of our high-order attention mechanism on the task of visual question answering (VQA), where we achieve state-of-the-art performance on the standard VQA dataset.