M4-RAG: A Massive-Scale Multilingual Multi-Cultural Multimodal RAG
This addresses the underexplored area of multilingual multimodal RAG for vision-language models, providing a foundation for advancing systems across languages and cultures, though it is incremental as it focuses on benchmarking and evaluation.
The paper tackles the problem of evaluating multilingual multimodal retrieval-augmented generation (RAG) for visual question answering by introducing M4-RAG, a massive-scale benchmark covering 42 languages and 56 dialects with over 80,000 culturally diverse image-question pairs, and finds that RAG consistently benefits smaller vision-language models but fails to scale to larger models, often degrading their performance.
Vision-language models (VLMs) have achieved strong performance in visual question answering (VQA), yet they remain constrained by static training data. Retrieval-Augmented Generation (RAG) mitigates this limitation by enabling access to up-to-date, culturally grounded, and multilingual information; however, multilingual multimodal RAG remains largely underexplored. We introduce M4-RAG, a massive-scale benchmark covering 42 languages and 56 regional dialects and registers, comprising over 80,000 culturally diverse image-question pairs for evaluating retrieval-augmented VQA across languages and modalities. To balance realism with reproducibility, we build a controlled retrieval environment containing millions of carefully curated multilingual documents relevant to the query domains, approximating real-world retrieval conditions while ensuring consistent experimentation. Our systematic evaluation reveals that although RAG consistently benefits smaller VLMs, it fails to scale to larger models and often even degrades their performance, exposing a critical mismatch between model size and current retrieval effectiveness. M4-RAG provides a foundation for advancing next-generation RAG systems capable of reasoning seamlessly across languages, modalities, and cultural contexts.