Tracking the Truth: Object-Centric Spatio-Temporal Monitoring for Video Large Language Models
For researchers and developers of video LLMs, this work addresses the critical problem of hallucination in dynamic scenes by providing a diagnostic benchmark and a novel object-centric framework.
Video LLMs hallucinate in dynamic scenes due to poor spatio-temporal monitoring. The authors introduce STEMO-Bench to diagnose this and propose STEMO-Track, an object-centric framework that reduces hallucinations and improves reasoning consistency over state-of-the-art models.
While multimodal large language models (MLLMs) have advanced video understanding, they remain highly prone to hallucinations in dynamic scenes. We argue this stems from a failure in spatio-temporal monitoring, the ability to persistently track object identities, states, and relations over time. Existing benchmarks obscure this deficit by relying on single final-answer evaluations for queries that can often be resolved via local visual cues or statistical priors. To rigorously diagnose this, we introduce STEMO-Bench (Spatio-TEmporal MOnitoring), a benchmark of human-verified object-centric facts that evaluates intermediate reasoning by decomposing queries into sub-questions, distinguishing genuine temporal understanding from coincidental correctness. To address failure modes exposed by STEMO, we propose STEMO-Track, a novel object-centric framework that explicitly constructs and reasons over structured object trajectories via chunk-wise state extraction and temporal aggregation. Extensive experiments demonstrate that our object-centric framework significantly reduces hallucinated answers and improves spatio-temporal reasoning consistency over state-of-the-art MLLMs.