On the Mutual Influence of Gender and Occupation in LLM Representations
This work addresses the problem of understanding and mitigating gender bias in LLMs for researchers and practitioners, but it is incremental as it builds on existing studies of bias in language models.
The researchers investigated how gender representations of first names in LLMs interact with occupational contexts, finding that these representations correlate with real-world gender statistics and are influenced by stereotypically gendered occupations. They also examined how these representations affect downstream occupation prediction tasks and their potential for bias detection, though reliable use for bias detection proved challenging.
We examine LLM representations of gender for first names in various occupational contexts to study how occupations and the gender perception of first names in LLMs influence each other mutually. We find that LLMs' first-name gender representations correlate with real-world gender statistics associated with the name, and are influenced by the co-occurrence of stereotypically feminine or masculine occupations. Additionally, we study the influence of first-name gender representations on LLMs in a downstream occupation prediction task and their potential as an internal metric to identify extrinsic model biases. While feminine first-name embeddings often raise the probabilities for female-dominated jobs (and vice versa for male-dominated jobs), reliably using these internal gender representations for bias detection remains challenging.