Behavioral Bias of Vision-Language Models: A Behavioral Finance View
This work addresses potential biases in LVLMs for applications in finance and psychology, but it is incremental as it applies existing behavioral finance concepts to a new model type.
The paper tackles the problem of behavioral biases in large vision-language models (LVLMs) by evaluating them from a behavioral finance perspective, finding that open-source models like LLaVA-NeXT and Mini-Gemini suffer significantly from recency and authority biases, while GPT-4o is minimally affected.
Large Vision-Language Models (LVLMs) evolve rapidly as Large Language Models (LLMs) was equipped with vision modules to create more human-like models. However, we should carefully evaluate their applications in different domains, as they may possess undesired biases. Our work studies the potential behavioral biases of LVLMs from a behavioral finance perspective, an interdisciplinary subject that jointly considers finance and psychology. We propose an end-to-end framework, from data collection to new evaluation metrics, to assess LVLMs' reasoning capabilities and the dynamic behaviors manifested in two established human financial behavioral biases: recency bias and authority bias. Our evaluations find that recent open-source LVLMs such as LLaVA-NeXT, MobileVLM-V2, Mini-Gemini, MiniCPM-Llama3-V 2.5 and Phi-3-vision-128k suffer significantly from these two biases, while the proprietary model GPT-4o is negligibly impacted. Our observations highlight directions in which open-source models can improve. The code is available at https://github.com/mydcxiao/vlm_behavioral_fin.