LogicGaze: Benchmarking Causal Consistency in Visual Narratives via Counterfactual Verification
This addresses the issue of hallucination in VLMs for multimodal AI applications, though it is incremental as it focuses on benchmarking rather than solving the problem.
The paper tackles the problem of unreliable sequential reasoning in Vision-Language Models (VLMs) by introducing LogicGaze, a benchmark that tests causal consistency in visual narratives, revealing significant vulnerabilities in models like Qwen2.5-VL-72B.
While sequential reasoning enhances the capability of Vision-Language Models (VLMs) to execute complex multimodal tasks, their reliability in grounding these reasoning chains within actual visual evidence remains insufficiently explored. We introduce LogicGaze, a novel benchmark framework designed to rigorously interrogate whether VLMs can validate sequential causal chains against visual inputs, specifically targeting the pervasive issue of hallucination. Curated from 40,000 video segments from ShareGPT4Video and a subset of Flickr30k imagery, LogicGaze integrates causal sequences with visually contradictory yet linguistically plausible perturbations, compelling models to verify the authenticity of each reasoning step. Our tripartite evaluation protocol - Causal Validation, Grounded Narrative Synthesis, and Perturbation Rejection - exposes significant vulnerabilities in state-of-the-art VLMs such as Qwen2.5-VL-72B. LogicGaze advocates for robust, trustworthy multimodal reasoning, with all resources publicly available in an anonymized repository.