QirK: Question Answering via Intermediate Representation on Knowledge Graphs
This addresses the challenge of reliable question answering on structured data for users needing accurate information from knowledge graphs, representing an incremental improvement by combining existing technologies.
The paper tackles the problem of answering structurally complex natural language questions on Knowledge Graphs, which are beyond current Large Language Models, by introducing QirK, a system that uses an intermediate representation to map questions to valid database queries, achieving practical synthesis of LLM capabilities and KG reliability.
We demonstrate QirK, a system for answering natural language questions on Knowledge Graphs (KG). QirK can answer structurally complex questions that are still beyond the reach of emerging Large Language Models (LLMs). It does so using a unique combination of database technology, LLMs, and semantic search over vector embeddings. The glue for these components is an intermediate representation (IR). The input question is mapped to IR using LLMs, which is then repaired into a valid relational database query with the aid of a semantic search on vector embeddings. This allows a practical synthesis of LLM capabilities and KG reliability. A short video demonstrating QirK is available at https://youtu.be/6c81BLmOZ0U.