Part-of-Speech Relevance Weights for Learning Word Embeddings
This addresses a limitation in word embedding models for natural language processing by integrating POS information, but it is incremental as it builds on existing methods.
The paper tackles the problem of learning word embeddings by incorporating part-of-speech (POS) relevance weights to model syntactic relationships, and experiments show effectiveness on analogy and similarity tasks.
This paper proposes a model to learn word embeddings with weighted contexts based on part-of-speech (POS) relevance weights. POS is a fundamental element in natural language. However, state-of-the-art word embedding models fail to consider it. This paper proposes to use position-dependent POS relevance weighting matrices to model the inherent syntactic relationship among words within a context window. We utilize the POS relevance weights to model each word-context pairs during the word embedding training process. The model proposed in this paper paper jointly optimizes word vectors and the POS relevance matrices. Experiments conducted on popular word analogy and word similarity tasks all demonstrated the effectiveness of the proposed method.