Intra-Fairness Dynamics: The Bias Spillover Effect in Targeted LLM Alignment
This addresses the issue of multidimensional fairness for LLM developers and users, highlighting a critical gap in current alignment methods, though it is incremental as it builds on existing bias spillover research in machine learning.
The study tackled the problem of bias spillover in targeted LLM fairness alignment, finding that improving fairness for one sensitive attribute can significantly degrade fairness for others, especially in ambiguous contexts, with physical appearance showing p<0.001 degradations across all models.
Conventional large language model (LLM) fairness alignment largely focuses on mitigating bias along single sensitive attributes, overlooking fairness as an inherently multidimensional and context-specific value. This approach risks creating systems that achieve narrow fairness metrics while exacerbating disparities along untargeted attributes, a phenomenon known as bias spillover. While extensively studied in machine learning, bias spillover remains critically underexplored in LLM alignment. In this work, we investigate how targeted gender alignment affects fairness across nine sensitive attributes in three state-of-the-art LLMs (Mistral 7B, Llama 3.1 8B, Qwen 2.5 7B). Using Direct Preference Optimization and the BBQ benchmark, we evaluate fairness under ambiguous and disambiguous contexts. Our findings reveal noticeable bias spillover: while aggregate results show improvements, context-aware analysis exposes significant degradations in ambiguous contexts, particularly for physical appearance ($p< 0.001$ across all models), sexual orientation, and disability status. We demonstrate that improving fairness along one attribute can inadvertently worsen disparities in others under uncertainty, highlighting the necessity of context-aware, multi-attribute fairness evaluation frameworks.