TallyQA: Answering Complex Counting Questions
This addresses the need for more sophisticated counting capabilities in AI systems for visual question answering, though it is incremental as it builds on existing relation network methods.
The paper tackles the problem of complex counting questions in visual question answering, which involve relationships and reasoning beyond simple object detection, by introducing TallyQA, the largest dataset for open-ended counting, and a new algorithm using relation networks with region proposals, achieving state-of-the-art results on TallyQA and HowMany-QA benchmarks.
Most counting questions in visual question answering (VQA) datasets are simple and require no more than object detection. Here, we study algorithms for complex counting questions that involve relationships between objects, attribute identification, reasoning, and more. To do this, we created TallyQA, the world's largest dataset for open-ended counting. We propose a new algorithm for counting that uses relation networks with region proposals. Our method lets relation networks be efficiently used with high-resolution imagery. It yields state-of-the-art results compared to baseline and recent systems on both TallyQA and the HowMany-QA benchmark.