The POLAR Framework: Polar Opposites Enable Interpretability of Pre-Trained Word Embeddings
This work addresses the need for interpretability in word embeddings for researchers and engineers, though it is incremental as it adds interpretability without fundamentally changing the embeddings.
The authors tackled the problem of making pre-trained word embeddings interpretable by introducing the POLAR framework, which transforms embeddings into a new space using semantic differentials (e.g., cold-hot) and achieves performance comparable to original embeddings on downstream tasks while aligning with human judgement.
We introduce POLAR - a framework that adds interpretability to pre-trained word embeddings via the adoption of semantic differentials. Semantic differentials are a psychometric construct for measuring the semantics of a word by analysing its position on a scale between two polar opposites (e.g., cold -- hot, soft -- hard). The core idea of our approach is to transform existing, pre-trained word embeddings via semantic differentials to a new "polar" space with interpretable dimensions defined by such polar opposites. Our framework also allows for selecting the most discriminative dimensions from a set of polar dimensions provided by an oracle, i.e., an external source. We demonstrate the effectiveness of our framework by deploying it to various downstream tasks, in which our interpretable word embeddings achieve a performance that is comparable to the original word embeddings. We also show that the interpretable dimensions selected by our framework align with human judgement. Together, these results demonstrate that interpretability can be added to word embeddings without compromising performance. Our work is relevant for researchers and engineers interested in interpreting pre-trained word embeddings.