Sequence-to-Sequence Knowledge Graph Completion and Question Answering
This addresses scalability and integration issues in knowledge graph applications, offering a more efficient and versatile approach for tasks like link prediction and question answering, though it builds on existing Transformer methods.
The authors tackled the problem of large model sizes and multi-stage pipelines in knowledge graph embedding (KGE) models by using an off-the-shelf encoder-decoder Transformer for sequence-to-sequence link prediction and question answering, achieving state-of-the-art results with up to 98% model size reduction and outperforming baselines on multiple datasets.
Knowledge graph embedding (KGE) models represent each entity and relation of a knowledge graph (KG) with low-dimensional embedding vectors. These methods have recently been applied to KG link prediction and question answering over incomplete KGs (KGQA). KGEs typically create an embedding for each entity in the graph, which results in large model sizes on real-world graphs with millions of entities. For downstream tasks these atomic entity representations often need to be integrated into a multi stage pipeline, limiting their utility. We show that an off-the-shelf encoder-decoder Transformer model can serve as a scalable and versatile KGE model obtaining state-of-the-art results for KG link prediction and incomplete KG question answering. We achieve this by posing KG link prediction as a sequence-to-sequence task and exchange the triple scoring approach taken by prior KGE methods with autoregressive decoding. Such a simple but powerful method reduces the model size up to 98% compared to conventional KGE models while keeping inference time tractable. After finetuning this model on the task of KGQA over incomplete KGs, our approach outperforms baselines on multiple large-scale datasets without extensive hyperparameter tuning.