Representation Learning on Hyper-Relational and Numeric Knowledge Graphs with Transformers
This work addresses a specific limitation in knowledge graph embedding for domains requiring numeric data, such as historical or scientific records, by extending hyper-relational models to incorporate numeric values, representing an incremental advancement.
The paper tackles the problem of learning representations for hyper-relational knowledge graphs that include numeric literals, proposing HyNT, a unified framework using transformers to handle both qualifiers and numeric information, which significantly outperforms state-of-the-art methods on real-world datasets.
A hyper-relational knowledge graph has been recently studied where a triplet is associated with a set of qualifiers; a qualifier is composed of a relation and an entity, providing auxiliary information for a triplet. While existing hyper-relational knowledge graph embedding methods assume that the entities are discrete objects, some information should be represented using numeric values, e.g., (J.R.R., was born in, 1892). Also, a triplet (J.R.R., educated at, Oxford Univ.) can be associated with a qualifier such as (start time, 1911). In this paper, we propose a unified framework named HyNT that learns representations of a hyper-relational knowledge graph containing numeric literals in either triplets or qualifiers. We define a context transformer and a prediction transformer to learn the representations based not only on the correlations between a triplet and its qualifiers but also on the numeric information. By learning compact representations of triplets and qualifiers and feeding them into the transformers, we reduce the computation cost of using transformers. Using HyNT, we can predict missing numeric values in addition to missing entities or relations in a hyper-relational knowledge graph. Experimental results show that HyNT significantly outperforms state-of-the-art methods on real-world datasets.