Probing Explicit and Implicit Gender Bias through LLM Conditional Text Generation
This addresses the challenge of comprehensively assessing gender bias in LLMs for AI fairness research, though it is incremental in extending evaluation beyond explicit stereotypes.
The study tackled the problem of evaluating gender bias in Large Language Models (LLMs) by proposing a conditional text generation mechanism that probes both explicit and implicit biases without predefined gender phrases or stereotypes, finding that all tested LLMs exhibited such biases regardless of model size.
Large Language Models (LLMs) can generate biased and toxic responses. Yet most prior work on LLM gender bias evaluation requires predefined gender-related phrases or gender stereotypes, which are challenging to be comprehensively collected and are limited to explicit bias evaluation. In addition, we believe that instances devoid of gender-related language or explicit stereotypes in inputs can still induce gender bias in LLMs. Thus, in this work, we propose a conditional text generation mechanism without the need for predefined gender phrases and stereotypes. This approach employs three types of inputs generated through three distinct strategies to probe LLMs, aiming to show evidence of explicit and implicit gender biases in LLMs. We also utilize explicit and implicit evaluation metrics to evaluate gender bias in LLMs under different strategies. Our experiments demonstrate that an increased model size does not consistently lead to enhanced fairness and all tested LLMs exhibit explicit and/or implicit gender bias, even when explicit gender stereotypes are absent in the inputs.