Domain-robust VQA with diverse datasets and methods but no target labels
This work addresses domain robustness for VQA systems, which is crucial for real-world applications but has been limited compared to object recognition, though it is incremental in adapting existing adaptation methods to VQA.
The paper tackled the problem of domain robustness in visual question answering (VQA) by quantifying domain shifts between datasets and testing the robustness of various VQA methods, resulting in the development of a new domain adaptation method adjusted for VQA models.
The observation that computer vision methods overfit to dataset specifics has inspired diverse attempts to make object recognition models robust to domain shifts. However, similar work on domain-robust visual question answering methods is very limited. Domain adaptation for VQA differs from adaptation for object recognition due to additional complexity: VQA models handle multimodal inputs, methods contain multiple steps with diverse modules resulting in complex optimization, and answer spaces in different datasets are vastly different. To tackle these challenges, we first quantify domain shifts between popular VQA datasets, in both visual and textual space. To disentangle shifts between datasets arising from different modalities, we also construct synthetic shifts in the image and question domains separately. Second, we test the robustness of different families of VQA methods (classic two-stream, transformer, and neuro-symbolic methods) to these shifts. Third, we test the applicability of existing domain adaptation methods and devise a new one to bridge VQA domain gaps, adjusted to specific VQA models. To emulate the setting of real-world generalization, we focus on unsupervised domain adaptation and the open-ended classification task formulation.