Think, Then Verify: A Hypothesis-Verification Multi-Agent Framework for Long Video Understanding
This work is significant for researchers and practitioners working on long video understanding, as it offers a novel approach to mitigate semantic drift and improve reasoning accuracy and interpretability.
This paper addresses the challenges of long video understanding, such as visual redundancy and semantic drift, by proposing VideoHV-Agent, a framework that reformulates video question answering as a hypothesis-verification process. It achieves state-of-the-art accuracy on three long-video understanding benchmarks while offering enhanced interpretability, improved logical soundness, and reduced computational cost.
Long video understanding is challenging due to dense visual redundancy, long-range temporal dependencies, and the tendency of chain-of-thought and retrieval-based agents to accumulate semantic drift and correlation-driven errors. We argue that long-video reasoning should begin not with reactive retrieval, but with deliberate task formulation: the model must first articulate what must be true in the video for each candidate answer to hold. This thinking-before-finding principle motivates VideoHV-Agent, a framework that reformulates video question answering as a structured hypothesis-verification process. Based on video summaries, a Thinker rewrites answer candidates into testable hypotheses, a Judge derives a discriminative clue specifying what evidence must be checked, a Verifier grounds and tests the clue using localized, fine-grained video content, and an Answer agent integrates validated evidence to produce the final answer. Experiments on three long-video understanding benchmarks show that VideoHV-Agent achieves state-of-the-art accuracy while providing enhanced interpretability, improved logical soundness, and lower computational cost. We make our code publicly available at: https://github.com/Haorane/VideoHV-Agent.