Evaluating Reasoning Faithfulness in Medical Vision-Language Models using Multimodal Perturbations
This addresses trust issues in high-stakes clinical applications by providing a framework to detect misaligned explanations, though it is incremental as it builds on existing evaluation methods.
The paper tackles the problem of evaluating whether chain-of-thought explanations from vision-language models in medical chest X-ray VQA faithfully reflect the decision process, finding that answer accuracy and explanation quality can be decoupled, with proprietary models outperforming open-source ones on attribution (25.0% vs. 1.4%) and fidelity (36.1% vs. 31.7%).
Vision-language models (VLMs) often produce chain-of-thought (CoT) explanations that sound plausible yet fail to reflect the underlying decision process, undermining trust in high-stakes clinical use. Existing evaluations rarely catch this misalignment, prioritizing answer accuracy or adherence to formats. We present a clinically grounded framework for chest X-ray visual question answering (VQA) that probes CoT faithfulness via controlled text and image modifications across three axes: clinical fidelity, causal attribution, and confidence calibration. In a reader study (n=4), evaluator-radiologist correlations fall within the observed inter-radiologist range for all axes, with strong alignment for attribution (Kendall's $τ_b=0.670$), moderate alignment for fidelity ($τ_b=0.387$), and weak alignment for confidence tone ($τ_b=0.091$), which we report with caution. Benchmarking six VLMs shows that answer accuracy and explanation quality can be decoupled, acknowledging injected cues does not ensure grounding, and text cues shift explanations more than visual cues. While some open-source models match final answer accuracy, proprietary models score higher on attribution (25.0% vs. 1.4%) and often on fidelity (36.1% vs. 31.7%), highlighting deployment risks and the need to evaluate beyond final answer accuracy.