Visualization Biases MLLM's Decision Making in Network Data Tasks
This highlights a practical risk for users of generative AI in network analysis, showing visualization-induced hallucinations.
The study investigated how visualizations bias MLLMs' decisions about bridge presence in networks, finding that visualizations increase confidence but create strong biases independent of actual bridge existence.
We evaluate how visualizations can influence the judgment of MLLMs about the presence or absence of bridges in a network. We show that the inclusion of visualization improves confidence over a structured text-based input that could theoretically be helpful for answering the question. On the other hand, we observe that standard visualization techniques create a strong bias towards accepting or refuting the presence of a bridge -- independently of whether or not a bridge actually exists in the network. While our results indicate that the inclusion of visualization techniques can effectively influence the MLLM's judgment without compromising its self-reported confidence, they also imply that practitioners must be careful of allowing users to include visualizations in generative AI applications so as to avoid undesired hallucinations.