Hadamard Product for Low-rank Bilinear Pooling
This addresses computational efficiency for visual question-answering tasks, though it appears incremental as it builds on existing bilinear pooling methods.
The paper tackles the high-dimensional computational limitations of bilinear models in multimodal learning by proposing low-rank bilinear pooling using Hadamard product, achieving state-of-the-art results on the VQA dataset with better parsimonious properties.
Bilinear models provide rich representations compared with linear models. They have been applied in various visual tasks, such as object recognition, segmentation, and visual question-answering, to get state-of-the-art performances taking advantage of the expanded representations. However, bilinear representations tend to be high-dimensional, limiting the applicability to computationally complex tasks. We propose low-rank bilinear pooling using Hadamard product for an efficient attention mechanism of multimodal learning. We show that our model outperforms compact bilinear pooling in visual question-answering tasks with the state-of-the-art results on the VQA dataset, having a better parsimonious property.