ZeroKBC: A Comprehensive Benchmark for Zero-Shot Knowledge Base Completion
This work addresses the challenge of handling emerging entities and relations in growing knowledge bases, which is an incremental step for AI systems relying on knowledge graphs.
The authors tackled the problem of zero-shot knowledge base completion (KBC) for unseen entities and relations, creating a comprehensive benchmark called ZeroKBC that reveals gaps in existing settings and shows that current state-of-the-art KBC systems perform poorly on it.
Knowledge base completion (KBC) aims to predict the missing links in knowledge graphs. Previous KBC tasks and approaches mainly focus on the setting where all test entities and relations have appeared in the training set. However, there has been limited research on the zero-shot KBC settings, where we need to deal with unseen entities and relations that emerge in a constantly growing knowledge base. In this work, we systematically examine different possible scenarios of zero-shot KBC and develop a comprehensive benchmark, ZeroKBC, that covers these scenarios with diverse types of knowledge sources. Our systematic analysis reveals several missing yet important zero-shot KBC settings. Experimental results show that canonical and state-of-the-art KBC systems cannot achieve satisfactory performance on this challenging benchmark. By analyzing the strength and weaknesses of these systems on solving ZeroKBC, we further present several important observations and promising future directions.