Believing without Seeing: Quality Scores for Contextualizing Vision-Language Model Explanations
This addresses the issue of fostering appropriate reliance on VLM predictions for users who cannot access visual contexts, such as blind and low-vision individuals, and is incremental as it builds on prior work on explanation reliability.
The paper tackles the problem of users overrelying on inaccurate predictions from Vision-Language Models (VLMs) when they cannot see visual contexts, by proposing two quality scoring functions (Visual Fidelity and Contrastiveness) for VLM-generated explanations. The result shows that showing these quality scores alongside explanations improves participants' accuracy at predicting VLM correctness by 11.1% and reduces false belief in incorrect predictions by 15.4% in user studies on A-OKVQA and VizWiz tasks.
When people query Vision-Language Models (VLMs) but cannot see the accompanying visual context (e.g. for blind and low-vision users), augmenting VLM predictions with natural language explanations can signal which model predictions are reliable. However, prior work has found that explanations can easily convince users that inaccurate VLM predictions are correct. To remedy undesirable overreliance on VLM predictions, we propose evaluating two complementary qualities of VLM-generated explanations via two quality scoring functions. We propose Visual Fidelity, which captures how faithful an explanation is to the visual context, and Contrastiveness, which captures how well the explanation identifies visual details that distinguish the model's prediction from plausible alternatives. On the A-OKVQA and VizWiz tasks, these quality scoring functions are better calibrated with model correctness than existing explanation qualities. We conduct a user study in which participants have to decide whether a VLM prediction is accurate without viewing its visual context. We observe that showing our quality scores alongside VLM explanations improves participants' accuracy at predicting VLM correctness by 11.1%, including a 15.4% reduction in the rate of falsely believing incorrect predictions. These findings highlight the utility of explanation quality scores in fostering appropriate reliance on VLM predictions.