Attributing Culture-Conditioned Generations to Pretraining Corpora
This addresses cultural bias in generative AI for diverse user groups, but it is incremental as it builds on existing work linking biases to pretraining data.
The paper tackles the problem of cultural bias in large language models by investigating how pretraining data patterns lead to biased culture-conditioned generations, finding that high-frequency cultures yield more memorized symbols while some low-frequency cultures produce none, and the model favors generating high-frequency entities regardless of cultural relevance.
In open-ended generative tasks like narrative writing or dialogue, large language models often exhibit cultural biases, showing limited knowledge and generating templated outputs for less prevalent cultures. Recent works show that these biases may stem from uneven cultural representation in pretraining corpora. This work investigates how pretraining leads to biased culture-conditioned generations by analyzing how models associate entities with cultures based on pretraining data patterns. We propose the MEMOed framework (MEMOrization from pretraining document) to determine whether a generation for a culture arises from memorization. Using MEMOed on culture-conditioned generations about food and clothing for 110 cultures, we find that high-frequency cultures in pretraining data yield more generations with memorized symbols, while some low-frequency cultures produce none. Additionally, the model favors generating entities with extraordinarily high frequency regardless of the conditioned culture, reflecting biases toward frequent pretraining terms irrespective of relevance. We hope that the MEMOed framework and our insights will inspire more works on attributing model performance on pretraining data.