Correlations between Word Vector Sets
This work addresses the need for efficient and effective similarity measures in natural language processing, offering a novel approach that is incremental in its application of correlation methods to word vector sets.
The paper tackled the problem of measuring semantic textual similarity by proposing a new paradigm that treats word embeddings as observations from random variables, achieving excellent results on standard benchmarks and outperforming many recent methods with faster and simpler implementations.
Similarity measures based purely on word embeddings are comfortably competing with much more sophisticated deep learning and expert-engineered systems on unsupervised semantic textual similarity (STS) tasks. In contrast to commonly used geometric approaches, we treat a single word embedding as e.g. 300 observations from a scalar random variable. Using this paradigm, we first illustrate that similarities derived from elementary pooling operations and classic correlation coefficients yield excellent results on standard STS benchmarks, outperforming many recently proposed methods while being much faster and trivial to implement. Next, we demonstrate how to avoid pooling operations altogether and compare sets of word embeddings directly via correlation operators between reproducing kernel Hilbert spaces. Just like cosine similarity is used to compare individual word vectors, we introduce a novel application of the centered kernel alignment (CKA) as a natural generalisation of squared cosine similarity for sets of word vectors. Likewise, CKA is very easy to implement and enjoys very strong empirical results.