A Joint Model for Word Embedding and Word Morphology
This work addresses the problem of improving word embeddings and morphological analysis for natural language processing, but it is incremental as it builds on existing character-level and embedding methods.
The paper tackles unsupervised morphological analysis and word embedding learning by jointly modeling morpheme segmentation and character-level composition, achieving comparable morpheme boundary recovery to dedicated analyzers and better syntactic analogy answering than word-based embedding models.
This paper presents a joint model for performing unsupervised morphological analysis on words, and learning a character-level composition function from morphemes to word embeddings. Our model splits individual words into segments, and weights each segment according to its ability to predict context words. Our morphological analysis is comparable to dedicated morphological analyzers at the task of morpheme boundary recovery, and also performs better than word-based embedding models at the task of syntactic analogy answering. Finally, we show that incorporating morphology explicitly into character-level models help them produce embeddings for unseen words which correlate better with human judgments.