Seeing but Not Believing: Probing the Disconnect Between Visual Attention and Answer Correctness in VLMs
This addresses reliability issues in VLMs for multimodal AI applications, though it is incremental as it builds on existing diagnostic understanding.
The study found that Vision-Language Models (VLMs) often perceive correct visual evidence but still output incorrect answers, a phenomenon termed 'seeing but not believing', and an inference-time intervention improved accuracy across multiple VLM families.
Vision-Language Models (VLMs) achieve strong results on multimodal tasks such as visual question answering, yet they can still fail even when the correct visual evidence is present. In this work, we systematically investigate whether these failures arise from not perceiving the evidence or from not leveraging it effectively. By examining layer-wise attention dynamics, we find that shallow layers focus primarily on text, while deeper layers sparsely but reliably attend to localized evidence regions. Surprisingly, VLMs often perceive the visual evidence when outputting incorrect answers, a phenomenon we term ``seeing but not believing'' that widely exists in major VLM families. Building on this, we introduce an inference-time intervention that highlights deep-layer evidence regions through selective attention-based masking. It requires no training and consistently improves accuracy across multiple families, including LLaVA, Qwen, Gemma, and InternVL. These results show that VLMs encode reliable evidence internally but under-utilize it, making such signals explicit can bridge the gap between perception and reasoning, advancing the diagnostic understanding and reliability of VLMs.