A Group Fairness Lens for Large Language Models
This work addresses fairness concerns for LLM deployment in social media contexts, offering a more comprehensive evaluation approach, though it is incremental in building on existing bias mitigation methods.
The paper tackled the problem of evaluating and mitigating biases in large language models (LLMs) by proposing a group fairness lens with a hierarchical schema and a new dataset, GFair, and introduced a chain-of-thought method, GF-Think, which demonstrated efficacy in reducing biases to achieve fairness.
The rapid advancement of large language models has revolutionized various applications but also raised crucial concerns about their potential to perpetuate biases and unfairness when deployed in social media contexts. Evaluating LLMs' potential biases and fairness has become crucial, as existing methods rely on limited prompts focusing on just a few groups, lacking a comprehensive categorical perspective. In this paper, we propose evaluating LLM biases from a group fairness lens using a novel hierarchical schema characterizing diverse social groups. Specifically, we construct a dataset, GFair, encapsulating target-attribute combinations across multiple dimensions. In addition, we introduce statement organization, a new open-ended text generation task, to uncover complex biases in LLMs. Extensive evaluations of popular LLMs reveal inherent safety concerns. To mitigate the biases of LLM from a group fairness perspective, we pioneer a novel chain-of-thought method GF-Think to mitigate biases of LLMs from a group fairness perspective. Experimental results demonstrate its efficacy in mitigating bias in LLMs to achieve fairness.