Cross-Language Bias Examination in Large Language Models
This work addresses bias assessment for multilingual LLMs, providing a foundation for equitable development, though it is incremental as it extends existing methods to new languages.
The study tackled the problem of bias in Large Language Models by introducing a multilingual evaluation framework, revealing substantial gaps in bias across languages, such as higher stereotype bias in Arabic and Spanish and contrasting patterns like age showing low explicit but high implicit bias.
This study introduces an innovative multilingual bias evaluation framework for assessing bias in Large Language Models, combining explicit bias assessment through the BBQ benchmark with implicit bias measurement using a prompt-based Implicit Association Test. By translating the prompts and word list into five target languages, English, Chinese, Arabic, French, and Spanish, we directly compare different types of bias across languages. The results reveal substantial gaps in bias across languages used in LLMs. For example, Arabic and Spanish consistently show higher levels of stereotype bias, while Chinese and English exhibit lower levels of bias. We also identify contrasting patterns across bias types. Age shows the lowest explicit bias but the highest implicit bias, emphasizing the importance of detecting implicit biases that are undetectable with standard benchmarks. These findings indicate that LLMs vary significantly across languages and bias dimensions. This study fills a key research gap by providing a comprehensive methodology for cross-lingual bias analysis. Ultimately, our work establishes a foundation for the development of equitable multilingual LLMs, ensuring fairness and effectiveness across diverse languages and cultures.