ViFP: A Framework for Visual False Positive Detection to Enhance Reasoning Reliability in VLMs
This addresses reasoning reliability issues in VLMs, which is crucial for applications requiring trustworthy AI, though it appears incremental as it builds on existing detection and correction approaches.
The paper tackles the problem of false positive reasoning in vision-language models, where models produce correct answers but follow incorrect reasoning paths, and proposes ViFP, a framework that detects and corrects these errors to improve reliability. Experiments on datasets like A-OKVQA show ViFP improves accuracy by up to 5.4% and reduces false positives, surpassing previous state-of-the-art methods.
During reasoning in vision-language models (VLMs), false positive (FP) reasoning occurs when a model produces the correct answer but follows an incorrect reasoning path, resulting in undermined reasoning reliability. Existing approaches mainly rely on prompt engineering, knowledge distillation or reinforcement learning to improve reasoning reliability, both of which require large amounts of high-quality data and thus limit practical applicability. Few approaches have focused on directly detecting and correcting FPs. To address these issues, we propose ViFP, a framework for Visual False Positive Detection to Enhance Reasoning Reliability in VLMs. ViFP builds effective reasoning paths through multi-turn QA and dynamically analyzes the consistency of the reasoning path to identify potential FPs. It also introduces a targeted reasoning chain correction mechanism to modify FP reasoning, thereby improving logical consistency and accuracy. Finally, we introduce a reliability evaluation metric, VoC, which integrates answer accuracy and the FP rate, providing a quantitative tool to assess whether a VLM not only answers correctly but also reasons reliably. Our experiments on closed-source VLMs show that ViFP consistently improves performance across three datasets: A-OKVQA, OK-VQA, and FVQA. On A-OKVQA, ViFP improves accuracy by up to 5.4%, surpassing the previous state-of-the-art by 4.3%, and significantly reduces the number of FPs, validating its benefits in enhancing reasoning reliability.