MoReVQA: Exploring Modular Reasoning Models for Video Question Answering
This addresses the problem of brittle behavior in modular methods for videoQA, offering a more robust solution for researchers and practitioners in video understanding.
The paper tackles video question answering by proposing a multi-stage modular reasoning framework, MoReVQA, which improves over prior work with state-of-the-art results on standard benchmarks like NExT-QA, iVQA, EgoSchema, and ActivityNet-QA.
This paper addresses the task of video question answering (videoQA) via a decomposed multi-stage, modular reasoning framework. Previous modular methods have shown promise with a single planning stage ungrounded in visual content. However, through a simple and effective baseline, we find that such systems can lead to brittle behavior in practice for challenging videoQA settings. Thus, unlike traditional single-stage planning methods, we propose a multi-stage system consisting of an event parser, a grounding stage, and a final reasoning stage in conjunction with an external memory. All stages are training-free, and performed using few-shot prompting of large models, creating interpretable intermediate outputs at each stage. By decomposing the underlying planning and task complexity, our method, MoReVQA, improves over prior work on standard videoQA benchmarks (NExT-QA, iVQA, EgoSchema, ActivityNet-QA) with state-of-the-art results, and extensions to related tasks (grounded videoQA, paragraph captioning).