Frequency-based Distortions in Contextualized Word Embeddings
This addresses biases in NLP models that can lead to societal implications, such as difficulty in differentiating between regions, and is incremental in analyzing geometric characteristics.
The paper tackled the problem of how word frequency in pre-training data distorts similarity metrics in contextualized BERT embeddings, revealing that these distortions cause over- or under-estimation of semantic similarity compared to human judgments and persist across models like BERT-Multilingual.
How does word frequency in pre-training data affect the behavior of similarity metrics in contextualized BERT embeddings? Are there systematic ways in which some word relationships are exaggerated or understated? In this work, we explore the geometric characteristics of contextualized word embeddings with two novel tools: (1) an identity probe that predicts the identity of a word using its embedding; (2) the minimal bounding sphere for a word's contextualized representations. Our results reveal that words of high and low frequency differ significantly with respect to their representational geometry. Such differences introduce distortions: when compared to human judgments, point estimates of embedding similarity (e.g., cosine similarity) can over- or under-estimate the semantic similarity of two words, depending on the frequency of those words in the training data. This has downstream societal implications: BERT-Base has more trouble differentiating between South American and African countries than North American and European ones. We find that these distortions persist when using BERT-Multilingual, suggesting that they cannot be easily fixed with additional data, which in turn introduces new distortions.