Watching the News: Towards VideoQA Models that can Read
This addresses the gap in VideoQA models that ignore textual information, which is crucial for contextual reasoning in domains like news analysis, though it is incremental in proposing a new dataset and task rather than a fundamental breakthrough.
The paper tackles the problem of Video Question Answering (VideoQA) by introducing a novel task that requires reading and understanding text in videos, specifically focusing on news videos, and they created the NewsVideoQA dataset with over 8,600 QA pairs on 3,000+ videos to demonstrate the limitations of existing methods.
Video Question Answering methods focus on commonsense reasoning and visual cognition of objects or persons and their interactions over time. Current VideoQA approaches ignore the textual information present in the video. Instead, we argue that textual information is complementary to the action and provides essential contextualisation cues to the reasoning process. To this end, we propose a novel VideoQA task that requires reading and understanding the text in the video. To explore this direction, we focus on news videos and require QA systems to comprehend and answer questions about the topics presented by combining visual and textual cues in the video. We introduce the ``NewsVideoQA'' dataset that comprises more than $8,600$ QA pairs on $3,000+$ news videos obtained from diverse news channels from around the world. We demonstrate the limitations of current Scene Text VQA and VideoQA methods and propose ways to incorporate scene text information into VideoQA methods.