A Deeper Look into Dependency-Based Word Embeddings
This work addresses the problem of optimizing word embeddings for NLP tasks, but it is incremental as it compares existing dependency variations without introducing a new method.
The study examined how different dependency-based word embeddings perform on distinguishing functional vs. domain similarity, word similarity rankings, and downstream tasks in English, finding that Universal and Stanford dependency contexts excel in different tasks and enhanced dependencies often boost performance.
We investigate the effect of various dependency-based word embeddings on distinguishing between functional and domain similarity, word similarity rankings, and two downstream tasks in English. Variations include word embeddings trained using context windows from Stanford and Universal dependencies at several levels of enhancement (ranging from unlabeled, to Enhanced++ dependencies). Results are compared to basic linear contexts and evaluated on several datasets. We found that embeddings trained with Universal and Stanford dependency contexts excel at different tasks, and that enhanced dependencies often improve performance.