A Representation-Level Assessment of Bias Mitigation in Foundation Models
This work addresses bias in foundation models for fairness and interpretability, offering an incremental tool for validating debiasing methods.
The study investigated how bias mitigation reshapes the embedding space of foundation models like BERT and Llama2, finding that it reduces gender-occupation disparities and leads to more neutral internal representations, with consistent results across model types.
We investigate how successful bias mitigation reshapes the embedding space of encoder-only and decoder-only foundation models, offering an internal audit of model behaviour through representational analysis. Using BERT and Llama2 as representative architectures, we assess the shifts in associations between gender and occupation terms by comparing baseline and bias-mitigated variants of the models. Our findings show that bias mitigation reduces gender-occupation disparities in the embedding space, leading to more neutral and balanced internal representations. These representational shifts are consistent across both model types, suggesting that fairness improvements can manifest as interpretable and geometric transformations. These results position embedding analysis as a valuable tool for understanding and validating the effectiveness of debiasing methods in foundation models. To further promote the assessment of decoder-only models, we introduce WinoDec, a dataset consisting of 4,000 sequences with gender and occupation terms, and release it to the general public. (https://github.com/winodec/wino-dec)