Gender-preserving Debiasing for Pre-trained Word Embeddings
This addresses bias in NLP applications for users affected by discriminatory AI, but it is incremental as it builds on prior debiasing work.
The paper tackles gender bias in pre-trained word embeddings by proposing a debiasing method that removes stereotypical biases while preserving non-discriminative gender-related information, achieving better performance than existing state-of-the-art methods on benchmark datasets.
Word embeddings learnt from massive text collections have demonstrated significant levels of discriminative biases such as gender, racial or ethnic biases, which in turn bias the down-stream NLP applications that use those word embeddings. Taking gender-bias as a working example, we propose a debiasing method that preserves non-discriminative gender-related information, while removing stereotypical discriminative gender biases from pre-trained word embeddings. Specifically, we consider four types of information: \emph{feminine}, \emph{masculine}, \emph{gender-neutral} and \emph{stereotypical}, which represent the relationship between gender vs. bias, and propose a debiasing method that (a) preserves the gender-related information in feminine and masculine words, (b) preserves the neutrality in gender-neutral words, and (c) removes the biases from stereotypical words. Experimental results on several previously proposed benchmark datasets show that our proposed method can debias pre-trained word embeddings better than existing SoTA methods proposed for debiasing word embeddings while preserving gender-related but non-discriminative information.