Defining and Evaluating Fair Natural Language Generation
This work addresses fairness issues in NLG for users affected by biased AI outputs, but it is incremental as it builds on existing bias evaluation methods.
The paper tackles gender bias in natural language generation by introducing a fairness framework and evaluating two state-of-the-art language models, providing empirical evidence that these models embed gender bias.
Our work focuses on the biases that emerge in the natural language generation (NLG) task of sentence completion. In this paper, we introduce a framework of fairness for NLG followed by an evaluation of gender biases in two state-of-the-art language models. Our analysis provides a theoretical formulation for biases in NLG and empirical evidence that existing language generation models embed gender bias.