Latent Relation Language Models
This work addresses language modeling for applications requiring entity and relation understanding, representing an incremental advancement over existing approaches.
The authors tackled the problem of language modeling by incorporating knowledge graph relations to jointly model words and entities, resulting in improved performance over baseline and previous knowledge-graph-based methods.
In this paper, we propose Latent Relation Language Models (LRLMs), a class of language models that parameterizes the joint distribution over the words in a document and the entities that occur therein via knowledge graph relations. This model has a number of attractive properties: it not only improves language modeling performance, but is also able to annotate the posterior probability of entity spans for a given text through relations. Experiments demonstrate empirical improvements over both a word-based baseline language model and a previous approach that incorporates knowledge graph information. Qualitative analysis further demonstrates the proposed model's ability to learn to predict appropriate relations in context.