Causally Testing Gender Bias in LLMs: A Case Study on Occupational Bias
This work addresses harmful biases in LLMs, which is a critical issue for fairness in AI applications, though it is incremental as it builds on existing bias measurement research.
The paper tackles the problem of measuring gender bias in large language models (LLMs) by introducing a causal formulation and proposing a benchmark called OccuGender to test occupational gender bias, finding that state-of-the-art open-source LLMs exhibit substantial bias.
Generated texts from large language models (LLMs) have been shown to exhibit a variety of harmful, human-like biases against various demographics. These findings motivate research efforts aiming to understand and measure such effects. This paper introduces a causal formulation for bias measurement in generative language models. Based on this theoretical foundation, we outline a list of desiderata for designing robust bias benchmarks. We then propose a benchmark called OccuGender, with a bias-measuring procedure to investigate occupational gender bias. We test several state-of-the-art open-source LLMs on OccuGender, including Llama, Mistral, and their instruction-tuned versions. The results show that these models exhibit substantial occupational gender bias. Lastly, we discuss prompting strategies for bias mitigation and an extension of our causal formulation to illustrate the generalizability of our framework. Our code and data https://github.com/chenyuen0103/gender-bias.