Improving VQA Reliability: A Dual-Assessment Approach with Self-Reflection and Cross-Model Verification
This addresses the issue of unreliable, overconfident answers in VQA systems for users relying on VLMs, representing a strong specific gain rather than a foundational advancement.
The paper tackled the problem of hallucinations in Vision-Language Models (VLMs) for Visual Question Answering (VQA), which undermine reliability, by proposing the DAVR framework that integrates self-reflection and cross-model verification, achieving a leading Φ100 score of 39.64 and 100-AUC of 97.22 in the Reliable VQA Challenge.
Vision-language models (VLMs) have demonstrated significant potential in Visual Question Answering (VQA). However, the susceptibility of VLMs to hallucinations can lead to overconfident yet incorrect answers, severely undermining answer reliability. To address this, we propose Dual-Assessment for VLM Reliability (DAVR), a novel framework that integrates Self-Reflection and Cross-Model Verification for comprehensive uncertainty estimation. The DAVR framework features a dual-pathway architecture: one pathway leverages dual selector modules to assess response reliability by fusing VLM latent features with QA embeddings, while the other deploys external reference models for factual cross-checking to mitigate hallucinations. Evaluated in the Reliable VQA Challenge at ICCV-CLVL 2025, DAVR achieves a leading $Φ_{100}$ score of 39.64 and a 100-AUC of 97.22, securing first place and demonstrating its effectiveness in enhancing the trustworthiness of VLM responses.