sense2vec - A Fast and Accurate Method for Word Sense Disambiguation In Neural Word Embeddings
This addresses the issue of multiple word meanings in NLP tasks, offering a practical solution for consuming algorithms, though it is incremental in improving existing embedding techniques.
The paper tackles the problem of word sense ambiguity in neural word embeddings by introducing a fast and accurate method for supervised disambiguation, resulting in over 8% average error reduction in unlabeled attachment scores for dependency parsing across six languages.
Neural word representations have proven useful in Natural Language Processing (NLP) tasks due to their ability to efficiently model complex semantic and syntactic word relationships. However, most techniques model only one representation per word, despite the fact that a single word can have multiple meanings or "senses". Some techniques model words by using multiple vectors that are clustered based on context. However, recent neural approaches rarely focus on the application to a consuming NLP algorithm. Furthermore, the training process of recent word-sense models is expensive relative to single-sense embedding processes. This paper presents a novel approach which addresses these concerns by modeling multiple embeddings for each word based on supervised disambiguation, which provides a fast and accurate way for a consuming NLP model to select a sense-disambiguated embedding. We demonstrate that these embeddings can disambiguate both contrastive senses such as nominal and verbal senses as well as nuanced senses such as sarcasm. We further evaluate Part-of-Speech disambiguated embeddings on neural dependency parsing, yielding a greater than 8% average error reduction in unlabeled attachment scores across 6 languages.