Konstruktor: A Strong Baseline for Simple Knowledge Graph Question Answering
This addresses errors in simple question answering for users relying on knowledge graphs, though it is incremental as it builds on existing methods.
The paper tackles the problem of answering simple knowledge graph questions, which even large language models struggle with, by introducing Konstruktor, an approach that integrates language models and knowledge graphs through entity extraction, relation prediction, and querying, achieving strong results on four datasets.
While being one of the most popular question types, simple questions such as "Who is the author of Cinderella?", are still not completely solved. Surprisingly, even the most powerful modern Large Language Models are prone to errors when dealing with such questions, especially when dealing with rare entities. At the same time, as an answer may be one hop away from the question entity, one can try to develop a method that uses structured knowledge graphs (KGs) to answer such questions. In this paper, we introduce Konstruktor - an efficient and robust approach that breaks down the problem into three steps: (i) entity extraction and entity linking, (ii) relation prediction, and (iii) querying the knowledge graph. Our approach integrates language models and knowledge graphs, exploiting the power of the former and the interpretability of the latter. We experiment with two named entity recognition and entity linking methods and several relation detection techniques. We show that for relation detection, the most challenging step of the workflow, a combination of relation classification/generation and ranking outperforms other methods. We report Konstruktor's strong results on four datasets.