MaXM: Towards Multilingual Visual Question Answering
This addresses the problem of limited multilingual VQA data for researchers, though it is incremental as it builds on existing translation and annotation methods.
The paper tackles the lack of multilingual visual question answering (VQA) resources by proposing a translation-based framework to generate data with less human annotation, resulting in MaXM, a test-only benchmark in 7 languages, and benchmarking existing models.
Visual Question Answering (VQA) has been primarily studied through the lens of the English language. Yet, tackling VQA in other languages in the same manner would require a considerable amount of resources. In this paper, we propose scalable solutions to multilingual visual question answering (mVQA), on both data and modeling fronts. We first propose a translation-based framework to mVQA data generation that requires much less human annotation efforts than the conventional approach of directly collection questions and answers. Then, we apply our framework to the multilingual captions in the Crossmodal-3600 dataset and develop an efficient annotation protocol to create MaXM, a test-only VQA benchmark in 7 diverse languages. Finally, we develop a simple, lightweight, and effective approach as well as benchmark state-of-the-art English and multilingual VQA models. We hope that our benchmark encourages further research on mVQA.