Unexplored flaws in multiple-choice VQA evaluations
This work highlights critical flaws in evaluation methods for MLLMs, which could mislead assessments of model capabilities, though it is incremental as it builds on prior findings about biases.
The paper identified unexplored biases in prompt formatting that affect the reliability of multiple-choice Visual Question Answering (VQA) evaluations for Multimodal Large Language Models (MLLMs), showing high sensitivity to minor changes across seven MLLMs and five datasets with 48 prompt variations.
Multimodal Large Language Models (MLLMs) demonstrate strong capabilities in handling image-text inputs. A common way to assess this ability is through multiple-choice Visual Question Answering (VQA). Earlier works have already revealed that these benchmarks are sensitive to answer choice order, a limitation that can be mitigated through careful design. Yet, we highlight additional, unexplored biases in prompt formatting that question the reliability of current MLLM evaluations. Specifically, we identify three key variation factors in prompt formatting and analyze their impact through a large-scale study involving $\mathbf{\text{seven}}$ MLLMs and $\mathbf{\text{five}}$ VQA datasets, spanning $\mathbf{48}$ distinct $\mathbf{\text{prompt format variations}}$. Our findings reveal that multiple-choice VQA is highly sensitive to minor prompt format changes, even when these changes are semantically neutral. We further demonstrate that these biases persist independently of known order biases or the MLLM's confidence in the correct answer. Finally, we demonstrate that existing bias mitigation strategies fail to address these newly identified biases.