A Probabilistic Framework for Learning Domain Specific Hierarchical Word Embeddings
This addresses the issue for NLP practitioners and researchers dealing with domain-specific text data, but it is incremental as it builds on existing embedding methods with a structured probabilistic approach.
The paper tackles the problem of standard word embedding models failing to represent domain-specific word meanings by proposing a method to learn domain-specific embeddings from hierarchically organized text, such as e-commerce reviews, and demonstrates its effectiveness on real-world datasets compared to state-of-the-art approaches.
The meaning of a word often varies depending on its usage in different domains. The standard word embedding models struggle to represent this variation, as they learn a single global representation for a word. We propose a method to learn domain-specific word embeddings, from text organized into hierarchical domains, such as reviews in an e-commerce website, where products follow a taxonomy. Our structured probabilistic model allows vector representations for the same word to drift away from each other for distant domains in the taxonomy, to accommodate its domain-specific meanings. By learning sets of domain-specific word representations jointly, our model can leverage domain relationships, and it scales well with the number of domains. Using large real-world review datasets, we demonstrate the effectiveness of our model compared to state-of-the-art approaches, in learning domain-specific word embeddings that are both intuitive to humans and benefit downstream NLP tasks.