Cross-Lingual Transfer of Cultural Knowledge: An Asymmetric Phenomenon
This addresses the problem of cultural bias in multilingual AI systems, particularly for low-resource languages, though it is incremental in explaining transfer mechanisms.
The study investigated how cultural knowledge transfers across languages in large language models, finding bidirectional transfer between English and high-resource languages but asymmetric transfer where low-resource languages primarily transfer to English with limited reverse flow.
Despite substantial research efforts evaluating how well large language models~(LLMs) handle global cultural diversity, the mechanisms behind their cultural knowledge acquisition, particularly in multilingual settings, remain unclear. We study this question by investigating how cultural knowledge transfers across languages during language adaptation of LLMs. We introduce an interpretable framework for studying this transfer, ensuring training data transparency and controlling transfer effects. Through a study of four non-Anglophonic cultures, we observe bidirectional cultural transfer between English and other high-resource languages, while low-resource languages primarily transfer knowledge to English with limited reverse flow. To explain this asymmetric phenomenon, we propose a frequency-based hypothesis: cultural knowledge appearing more frequently in the pretraining data transfers more easily, which is supported by empirical analysis of the training corpora.