EtiCor++: Towards Understanding Etiquettical Bias in LLMs
This addresses the problem of cultural bias in LLMs for researchers and developers, but it is incremental as it builds on existing work in cultural sensitivity.
The authors tackled the lack of resources for evaluating cultural sensitivity in LLMs by introducing EtiCor++, a corpus of worldwide etiquettes, and found inherent bias towards certain regions in LLMs through extensive experimentation.
In recent years, researchers have started analyzing the cultural sensitivity of LLMs. In this respect, Etiquettes have been an active area of research. Etiquettes are region-specific and are an essential part of the culture of a region; hence, it is imperative to make LLMs sensitive to etiquettes. However, there needs to be more resources in evaluating LLMs for their understanding and bias with regard to etiquettes. In this resource paper, we introduce EtiCor++, a corpus of etiquettes worldwide. We introduce different tasks for evaluating LLMs for knowledge about etiquettes across various regions. Further, we introduce various metrics for measuring bias in LLMs. Extensive experimentation with LLMs shows inherent bias towards certain regions.