Distributional Models and Deep Learning Embeddings: Combining the Best of Both Worlds
This work addresses the challenge of enhancing word embeddings for natural language processing, but it is incremental as it builds on existing methods.
The paper tackles the problem of improving word representation by combining low-dimensional deep learning embeddings with high-dimensional distributional models, resulting in better performance on a word relatedness judgment task.
There are two main approaches to the distributed representation of words: low-dimensional deep learning embeddings and high-dimensional distributional models, in which each dimension corresponds to a context word. In this paper, we combine these two approaches by learning embeddings based on distributional-model vectors - as opposed to one-hot vectors as is standardly done in deep learning. We show that the combined approach has better performance on a word relatedness judgment task.