Quantifying and Reducing Stereotypes in Word Embeddings
This addresses the issue of bias amplification in AI applications for users affected by stereotypes, representing an incremental improvement with a novel method for a known bottleneck.
The paper tackled the problem of gender stereotypes in word embeddings, showing that embeddings contain significant gender stereotypes, especially regarding professions, and developed an efficient algorithm that reduces these stereotypes while preserving the embedding's geometric properties.
Machine learning algorithms are optimized to model statistical properties of the training data. If the input data reflects stereotypes and biases of the broader society, then the output of the learning algorithm also captures these stereotypes. In this paper, we initiate the study of gender stereotypes in {\em word embedding}, a popular framework to represent text data. As their use becomes increasingly common, applications can inadvertently amplify unwanted stereotypes. We show across multiple datasets that the embeddings contain significant gender stereotypes, especially with regard to professions. We created a novel gender analogy task and combined it with crowdsourcing to systematically quantify the gender bias in a given embedding. We developed an efficient algorithm that reduces gender stereotype using just a handful of training examples while preserving the useful geometric properties of the embedding. We evaluated our algorithm on several metrics. While we focus on male/female stereotypes, our framework may be applicable to other types of embedding biases.