Word Sense Induction with Neural biLM and Symmetric Patterns
This work addresses word sense disambiguation for natural language processing applications, representing an incremental advancement with specific gains.
The authors tackled the problem of Word Sense Induction by replacing an n-gram language model with a recurrent neural network and introducing dynamic symmetric patterns, achieving a large margin improvement over the state-of-the-art on the SemEval 2013 shared task.
An established method for Word Sense Induction (WSI) uses a language model to predict probable substitutes for target words, and induces senses by clustering these resulting substitute vectors. We replace the ngram-based language model (LM) with a recurrent one. Beyond being more accurate, the use of the recurrent LM allows us to effectively query it in a creative way, using what we call dynamic symmetric patterns. The combination of the RNN-LM and the dynamic symmetric patterns results in strong substitute vectors for WSI, allowing to surpass the current state-of-the-art on the SemEval 2013 WSI shared task by a large margin.