Bayesian Hierarchical Words Representation Learning
This work addresses the challenge of enhancing word embeddings for natural language processing tasks, particularly for rare words, but it is incremental as it builds on existing Bayesian and hierarchical methods.
The paper tackles the problem of learning word representations by introducing BHWR, which combines Variational Bayes with hierarchical priors to incorporate semantic taxonomy, resulting in improved representation quality and better performance for rare words compared to Bayesian alternatives.
This paper presents the Bayesian Hierarchical Words Representation (BHWR) learning algorithm. BHWR facilitates Variational Bayes word representation learning combined with semantic taxonomy modeling via hierarchical priors. By propagating relevant information between related words, BHWR utilizes the taxonomy to improve the quality of such representations. Evaluation of several linguistic datasets demonstrates the advantages of BHWR over suitable alternatives that facilitate Bayesian modeling with or without semantic priors. Finally, we further show that BHWR produces better representations for rare words.