Fairness through Difference Awareness: Measuring Desired Group Discrimination in LLMs
This work addresses fairness in LLMs for applications requiring contextual group differentiation, offering a novel benchmark but with incremental methodological contributions.
The paper tackles the problem of algorithmic fairness by arguing that group difference awareness is necessary in contexts like legal and harm assessments, and introduces a benchmark suite of 16k questions to measure this, showing that existing bias mitigation strategies can backfire on this dimension.
Algorithmic fairness has conventionally adopted the mathematically convenient perspective of racial color-blindness (i.e., difference unaware treatment). However, we contend that in a range of important settings, group difference awareness matters. For example, differentiating between groups may be necessary in legal contexts (e.g., the U.S. compulsory draft applies to men but not women) and harm assessments (e.g., referring to girls as ``terrorists'' may be less harmful than referring to Muslim people as such). Thus, in contrast to most fairness work, we study fairness through the perspective of treating people differently -- when it is contextually appropriate to. We first introduce an important distinction between descriptive (fact-based), normative (value-based), and correlation (association-based) benchmarks. This distinction is significant because each category requires separate interpretation and mitigation tailored to its specific characteristics. Then, we present a benchmark suite composed of eight different scenarios for a total of 16k questions that enables us to assess difference awareness. Finally, we show results across ten models that demonstrate difference awareness is a distinct dimension to fairness where existing bias mitigation strategies may backfire.