Mind the Language Gap: Automated and Augmented Evaluation of Bias in LLMs for High- and Low-Resource Languages
This work addresses bias evaluation for LLMs in diverse languages, which is crucial for fairness in NLP applications, though it appears incremental as it builds on prior methods.
The paper tackles the problem of social bias in large language models by introducing MLA-BiTe, a framework for systematic multilingual bias testing, and finds that it effectively evaluates bias across six languages, including low-resource ones, in seven sensitive categories.
Large Language Models (LLMs) have exhibited impressive natural language processing capabilities but often perpetuate social biases inherent in their training data. To address this, we introduce MultiLingual Augmented Bias Testing (MLA-BiTe), a framework that improves prior bias evaluation methods by enabling systematic multilingual bias testing. MLA-BiTe leverages automated translation and paraphrasing techniques to support comprehensive assessments across diverse linguistic settings. In this study, we evaluate the effectiveness of MLA-BiTe by testing four state-of-the-art LLMs in six languages -- including two low-resource languages -- focusing on seven sensitive categories of discrimination.