The Woman Worked as a Babysitter: On Biases in Language Generation
This addresses biases in AI-generated text, which can perpetuate stereotypes and harm marginalized groups, though it is incremental in refining bias measurement methods.
The paper tackles the problem of biases in natural language generation by analyzing text from language models for different demographic groups, introducing 'regard' as a bias metric and showing that sentiment scores are a poor proxy for it, with manual annotations and an automatic classifier revealing significant biases.
We present a systematic study of biases in natural language generation (NLG) by analyzing text generated from prompts that contain mentions of different demographic groups. In this work, we introduce the notion of the regard towards a demographic, use the varying levels of regard towards different demographics as a defining metric for bias in NLG, and analyze the extent to which sentiment scores are a relevant proxy metric for regard. To this end, we collect strategically-generated text from language models and manually annotate the text with both sentiment and regard scores. Additionally, we build an automatic regard classifier through transfer learning, so that we can analyze biases in unseen text. Together, these methods reveal the extent of the biased nature of language model generations. Our analysis provides a study of biases in NLG, bias metrics and correlated human judgments, and empirical evidence on the usefulness of our annotated dataset.