FairMonitor: A Four-Stage Automatic Framework for Detecting Stereotypes and Biases in Large Language Models
This work addresses fairness issues in LLMs for applications like education, but it is incremental as it builds on existing bias detection methods by adding interpretability and real-world testing.
The paper tackles the problem of detecting stereotypes and biases in Large Language Models (LLMs) by introducing a four-stage framework that directly evaluates generated content, using the education sector as a case study with 12,632 questions; experimental results show varying biases in five LLMs and high correlation with human annotations.
Detecting stereotypes and biases in Large Language Models (LLMs) can enhance fairness and reduce adverse impacts on individuals or groups when these LLMs are applied. However, the majority of existing methods focus on measuring the model's preference towards sentences containing biases and stereotypes within datasets, which lacks interpretability and cannot detect implicit biases and stereotypes in the real world. To address this gap, this paper introduces a four-stage framework to directly evaluate stereotypes and biases in the generated content of LLMs, including direct inquiry testing, serial or adapted story testing, implicit association testing, and unknown situation testing. Additionally, the paper proposes multi-dimensional evaluation metrics and explainable zero-shot prompts for automated evaluation. Using the education sector as a case study, we constructed the Edu-FairMonitor based on the four-stage framework, which encompasses 12,632 open-ended questions covering nine sensitive factors and 26 educational scenarios. Experimental results reveal varying degrees of stereotypes and biases in five LLMs evaluated on Edu-FairMonitor. Moreover, the results of our proposed automated evaluation method have shown a high correlation with human annotations.