Beyond Words: Exploring Cultural Value Sensitivity in Multimodal Models
This addresses the problem of cultural bias in multimodal models for AI researchers and developers, though it is incremental as it extends existing LLM research to VLMs.
The paper investigates cultural value alignment in large vision-language models (VLMs), finding that they exhibit sensitivity to cultural values similar to LLMs, but performance is highly context-dependent, with alignment varying significantly across contexts.
Investigating value alignment in Large Language Models (LLMs) based on cultural context has become a critical area of research. However, similar biases have not been extensively explored in large vision-language models (VLMs). As the scale of multimodal models continues to grow, it becomes increasingly important to assess whether images can serve as reliable proxies for culture and how these values are embedded through the integration of both visual and textual data. In this paper, we conduct a thorough evaluation of multimodal model at different scales, focusing on their alignment with cultural values. Our findings reveal that, much like LLMs, VLMs exhibit sensitivity to cultural values, but their performance in aligning with these values is highly context-dependent. While VLMs show potential in improving value understanding through the use of images, this alignment varies significantly across contexts highlighting the complexities and underexplored challenges in the alignment of multimodal models.