Linguistic Nepotism: Trading-off Quality for Language Preference in Multilingual RAG
This reveals unintended biases in multilingual AI systems, which could affect users relying on citations for accuracy, though it is incremental in highlighting a specific issue.
The study investigated whether multilingual RAG systems exhibit language bias in citation choices, finding that models preferentially cite English sources for English queries, especially for lower-resource languages, and sometimes trade document relevance for language preference.
Multilingual Retrieval-Augmented Generation (mRAG) systems enable language models to answer knowledge-intensive queries with citation-supported responses across languages. While such systems have been proposed, an open questions is whether the mixture of different document languages impacts generation and citation in unintended ways. To investigate, we introduce a controlled methodology using model internals to measure language preference while holding other factors such as document relevance constant. Across eight languages and six open-weight models, we find that models preferentially cite English sources when queries are in English, with this bias amplified for lower-resource languages and for documents positioned mid-context. Crucially, we find that models sometimes trade-off document relevance for language preference, indicating that citation choices are not always driven by informativeness alone. Our findings shed light on how language models leverage multilingual context and influence citation behavior.