Benchmarking neural embeddings for link prediction in knowledge graphs under semantic and structural changes
This work addresses the need for robust evaluation methods for knowledge graph completion algorithms, particularly for frequently updated graphs, but it is incremental as it focuses on benchmarking rather than new algorithmic advances.
The authors tackled the problem of evaluating neural embeddings for link prediction in knowledge graphs under semantic and structural changes by proposing an open-source evaluation pipeline that benchmarks accuracy and connects it to graph structure, but no concrete performance numbers are provided.
Recently, link prediction algorithms based on neural embeddings have gained tremendous popularity in the Semantic Web community, and are extensively used for knowledge graph completion. While algorithmic advances have strongly focused on efficient ways of learning embeddings, fewer attention has been drawn to the different ways their performance and robustness can be evaluated. In this work we propose an open-source evaluation pipeline, which benchmarks the accuracy of neural embeddings in situations where knowledge graphs may experience semantic and structural changes. We define relation-centric connectivity measures that allow us to connect the link prediction capacity to the structure of the knowledge graph. Such an evaluation pipeline is especially important to simulate the accuracy of embeddings for knowledge graphs that are expected to be frequently updated.