Linking GloVe with word2vec
This work clarifies theoretical connections for researchers in natural language processing, but it is incremental as it builds on existing models without introducing new methods.
The paper tackles the problem of understanding the relationship between two popular word embedding methods, GloVe and word2vec's SGNS, by showing that their training objectives are similar despite different cost functions.
The Global Vectors for word representation (GloVe), introduced by Jeffrey Pennington et al. is reported to be an efficient and effective method for learning vector representations of words. State-of-the-art performance is also provided by skip-gram with negative-sampling (SGNS) implemented in the word2vec tool. In this note, we explain the similarities between the training objectives of the two models, and show that the objective of SGNS is similar to the objective of a specialized form of GloVe, though their cost functions are defined differently.