Word Embedding with Neural Probabilistic Prior
This work addresses the need for better word embeddings in natural language processing, but it appears incremental as it builds on existing models with a new regularization technique.
The authors tackled the problem of improving word representation learning by proposing a probabilistic prior that integrates with existing word embedding models, resulting in enhanced representation, robustness, and stability as demonstrated in experiments.
To improve word representation learning, we propose a probabilistic prior which can be seamlessly integrated with word embedding models. Different from previous methods, word embedding is taken as a probabilistic generative model, and it enables us to impose a prior regularizing word representation learning. The proposed prior not only enhances the representation of embedding vectors but also improves the model's robustness and stability. The structure of the proposed prior is simple and effective, and it can be easily implemented and flexibly plugged in most existing word embedding models. Extensive experiments show the proposed method improves word representation on various tasks.