Anchoring LLM Gender Bias to Human Baselines: A Cross-Lingual Audit
This research addresses the problem of quantifying and characterizing gender bias in LLMs, particularly its cross-lingual variability, for developers and users of these models. It is an incremental contribution to the field of LLM bias auditing.
This paper audited six large language models (LLMs) for gender stereotyping across four languages (English, Korean, Chinese, Japanese) using the HEXACO-100 personality inventory and anchoring against a cross-cultural human dataset. The study found that LLM stereotyping spans a range 2.5 times wider than human cross-country variation, and one English-centric model prompted in Korean reached 5 times the local human baseline.
We audit six large language models (LLMs) for gender stereotyping across English, Korean, Chinese, and Japanese. Three were developed primarily for English-language use (Claude, GPT, Gemini) and three for East Asian use (DeepSeek, Syn-Pro, HyperCLOVA X). We adopt the HEXACO-100 personality inventory and anchor each model against a cross-cultural human dataset spanning 48 countries to ask not whether LLMs are biased, but how far their gender attributions drift from the populations they are deployed among. Our findings show that their stereotyping spans a range roughly 2.5 times wider than the entire cross-country range found in humans, and the effect can compound across languages. One English-centric model, prompted in Korean, reached 5 times the local baseline, even when the prompt stated the candidate had already been hired, which often dampens human stereotyping. To characterize such behaviors without ranking them, we introduce a four-pattern framework -- concordance, suppression, reorganization, and amplification -- across 24 (model x language) cells. Item-level analysis reveals that translation does not just rescale stereotypes, but changes the attributes tied to it, hiding significant rearrangement under the surface while appearing well-calibrated. Our results ultimately suggest that no single debiasing pipeline is likely to address bias evenly across linguistic boundaries.