Interpretable Word Embeddings via Informative Priors
This addresses the lack of interpretability in word embeddings for computational social science and digital humanities, but it is incremental as it builds on existing probabilistic embedding methods.
The authors tackled the problem of interpretability and domain-specificity in word embeddings by proposing the use of informative priors to create interpretable and domain-informed dimensions, resulting in performance that captures latent semantic concepts better than or on-par with the current state of the art.
Word embeddings have demonstrated strong performance on NLP tasks. However, lack of interpretability and the unsupervised nature of word embeddings have limited their use within computational social science and digital humanities. We propose the use of informative priors to create interpretable and domain-informed dimensions for probabilistic word embeddings. Experimental results show that sensible priors can capture latent semantic concepts better than or on-par with the current state of the art, while retaining the simplicity and generalizability of using priors.