Detecting Clinical Hallucinations in LVLMs via Counterfactual Visual Grounding Uncertainty
For clinicians and AI safety researchers, this provides an interpretable and model-agnostic method to detect hallucinations in medical LVLMs, addressing a critical reliability issue.
The paper tackles hallucination detection in large vision-language models (LVLMs) for clinical image understanding. It proposes a framework that uses counterfactual visual grounding uncertainty to detect hallucinations, achieving consistent improvements over baselines across multiple medical imaging modalities and LVLM backbones.
Large vision-language models (LVLMs) are increasingly used for clinical image understanding, yet they remain vulnerable to \emph{hallucinations}--producing textual findings or attributes not supported by the image. We present a vision-traceable hallucination detection framework that audits arbitrary LVLM responses via visual evidence grounding, requiring neither modification nor internal access to the hidden states of LVLMs. Given an LVLM response, we extract visually verifiable entities and use a medical-domain-adapted Qwen-VL grounding verifier to localize each entity on the input image. To enhance the robustness of our detection method, we introduce a counterfactual entity perturbation method and estimate visual evidence uncertainty by contrasting factual and counterfactual grounding results. Specifically, we compute an entity-level uncertainty score from the positive confidence, counterfactual confidence, and their grounding overlap for binary hallucination decision-making. Experiments on multiple medical imaging modalities and LVLM backbones demonstrate that our method consistently improves hallucination detection performance over recent baselines, while providing interpretable localization evidence and strong cross-model transferability. Code and dataset are available at https://github.com/Agentic-CliniAI/CounterVHD.