Latent Variable Models for Visual Question Answering
This work addresses the challenge of enhancing question-answering accuracy in AI systems for multimodal tasks, though it is incremental as it builds on existing VQA methods by adding latent variables.
The paper tackles the problem of Visual Question Answering by incorporating extra information like captions and answer categories as latent variables during training, which improves performance on the VQA v2.0 dataset over strong baselines, especially those without extensive pre-training.
Current work on Visual Question Answering (VQA) explore deterministic approaches conditioned on various types of image and question features. We posit that, in addition to image and question pairs, other modalities are useful for teaching machine to carry out question answering. Hence in this paper, we propose latent variable models for VQA where extra information (e.g. captions and answer categories) are incorporated as latent variables, which are observed during training but in turn benefit question-answering performance at test time. Experiments on the VQA v2.0 benchmarking dataset demonstrate the effectiveness of our proposed models: they improve over strong baselines, especially those that do not rely on extensive language-vision pre-training.