Modeling Semantic Relatedness using Global Relation Vectors
This addresses the challenge of capturing rich relational information in natural language processing, though it appears incremental as it builds on existing word embedding techniques.
The paper tackles the problem of modeling semantic relationships between words by introducing a method to directly learn relation vectors from co-occurrence statistics, resulting in a variant of GloVe that embeds these vectors into the word vector space.
Word embedding models such as GloVe rely on co-occurrence statistics from a large corpus to learn vector representations of word meaning. These vectors have proven to capture surprisingly fine-grained semantic and syntactic information. While we may similarly expect that co-occurrence statistics can be used to capture rich information about the relationships between different words, existing approaches for modeling such relationships have mostly relied on manipulating pre-trained word vectors. In this paper, we introduce a novel method which directly learns relation vectors from co-occurrence statistics. To this end, we first introduce a variant of GloVe, in which there is an explicit connection between word vectors and PMI weighted co-occurrence vectors. We then show how relation vectors can be naturally embedded into the resulting vector space.