word2vec Skip-Gram with Negative Sampling is a Weighted Logistic PCA
arXiv:1705.09755v121 citations
Originality Synthesis-oriented
AI Analysis
This provides theoretical insight for researchers in natural language processing, but it is incremental as it reinterprets an existing method.
The paper demonstrates that the skip-ram formulation of word2vec with negative sampling is equivalent to weighted logistic PCA, enabling improved understanding and extension to higher-dimensional models.
We show that the skip-gram formulation of word2vec trained with negative sampling is equivalent to a weighted logistic PCA. This connection allows us to better understand the objective, compare it to other word embedding methods, and extend it to higher dimensional models.