Repurposing Knowledge Graph Embeddings for Triple Representation via Weak Supervision
This work addresses a specific bottleneck in knowledge graph representation for researchers and practitioners, offering an incremental improvement over existing methods.
The paper tackles the problem of lossy aggregation in knowledge graph embeddings by proposing a novel fine-tuning approach that uses weak supervision signals from pre-trained embeddings to learn triple representations, showing consistent improvement over state-of-the-art methods on triple classification and clustering tasks.
The majority of knowledge graph embedding techniques treat entities and predicates as separate embedding matrices, using aggregation functions to build a representation of the input triple. However, these aggregations are lossy, i.e. they do not capture the semantics of the original triples, such as information contained in the predicates. To combat these shortcomings, current methods learn triple embeddings from scratch without utilizing entity and predicate embeddings from pre-trained models. In this paper, we design a novel fine-tuning approach for learning triple embeddings by creating weak supervision signals from pre-trained knowledge graph embeddings. We develop a method for automatically sampling triples from a knowledge graph and estimating their pairwise similarities from pre-trained embedding models. These pairwise similarity scores are then fed to a Siamese-like neural architecture to fine-tune triple representations. We evaluate the proposed method on two widely studied knowledge graphs and show consistent improvement over other state-of-the-art triple embedding methods on triple classification and triple clustering tasks.