Syntax-Aware Multi-Sense Word Embeddings for Deep Compositional Models of Meaning
This work addresses the challenge of enhancing semantic representation in NLP for tasks like sentence-level analysis, though it appears incremental as it builds on existing compositional models.
The paper tackled the problem of improving deep compositional models of meaning in NLP by developing a framework that uses syntax-aware multi-sense word embeddings, dynamically selecting senses during composition, and reported results within the state-of-the-art range on the MSRPar task.
Deep compositional models of meaning acting on distributional representations of words in order to produce vectors of larger text constituents are evolving to a popular area of NLP research. We detail a compositional distributional framework based on a rich form of word embeddings that aims at facilitating the interactions between words in the context of a sentence. Embeddings and composition layers are jointly learned against a generic objective that enhances the vectors with syntactic information from the surrounding context. Furthermore, each word is associated with a number of senses, the most plausible of which is selected dynamically during the composition process. We evaluate the produced vectors qualitatively and quantitatively with positive results. At the sentence level, the effectiveness of the framework is demonstrated on the MSRPar task, for which we report results within the state-of-the-art range.