Probabilistic Relation Induction in Vector Space Embeddings
This work addresses a key challenge in natural language processing for researchers and practitioners by improving the reliability of extracting implicit relational knowledge from word embeddings, though it is incremental as it builds on existing translation-based methods.
The paper tackles the problem of extracting relational knowledge from word embeddings by proposing two probabilistic models, one based on relations-as-translations and another on a linear relationship assumption, resulting in more accurate predictions and clearer extraction capabilities.
Word embeddings have been found to capture a surprisingly rich amount of syntactic and semantic knowledge. However, it is not yet sufficiently well-understood how the relational knowledge that is implicitly encoded in word embeddings can be extracted in a reliable way. In this paper, we propose two probabilistic models to address this issue. The first model is based on the common relations-as-translations view, but is cast in a probabilistic setting. Our second model is based on the much weaker assumption that there is a linear relationship between the vector representations of related words. Compared to existing approaches, our models lead to more accurate predictions, and they are more explicit about what can and cannot be extracted from the word embedding.