First-Person Fairness in Chatbots
This work addresses fairness for chatbot users, providing a practical methodology for bias monitoring and mitigation in an incremental advancement over existing fairness evaluation methods.
The paper tackled the problem of evaluating fairness in chatbots, which lack established methods due to their diverse and open-ended tasks, by introducing a scalable counterfactual approach to assess 'first-person fairness' based on demographic characteristics, resulting in the first large-scale evaluation using real-world chat data across millions of interactions and showing that post-training reinforcement learning techniques significantly mitigate biases.
Evaluating chatbot fairness is crucial given their rapid proliferation, yet typical chatbot tasks (e.g., resume writing, entertainment) diverge from the institutional decision-making tasks (e.g., resume screening) which have traditionally been central to discussion of algorithmic fairness. The open-ended nature and diverse use-cases of chatbots necessitate novel methods for bias assessment. This paper addresses these challenges by introducing a scalable counterfactual approach to evaluate "first-person fairness," meaning fairness toward chatbot users based on demographic characteristics. Our method employs a Language Model as a Research Assistant (LMRA) to yield quantitative measures of harmful stereotypes and qualitative analyses of demographic differences in chatbot responses. We apply this approach to assess biases in six of our language models across millions of interactions, covering sixty-six tasks in nine domains and spanning two genders and four races. Independent human annotations corroborate the LMRA-generated bias evaluations. This study represents the first large-scale fairness evaluation based on real-world chat data. We highlight that post-training reinforcement learning techniques significantly mitigate these biases. This evaluation provides a practical methodology for ongoing bias monitoring and mitigation.