UltRAG: a Universal Simple Scalable Recipe for Knowledge Graph RAG
This addresses the challenge of adapting RAG to knowledge graphs for more accurate question answering, representing a novel method rather than an incremental improvement.
The paper tackles the problem of factual errors in LLMs when handling knowledge graph queries requiring multi-hop reasoning, introducing ULTRAG which achieves state-of-the-art results on KGQA tasks and scales to Wikidata-size graphs with 116M entities and 1.6B relations.
Large language models (LLMs) frequently generate confident yet factually incorrect content when used for language generation (a phenomenon often known as hallucination). Retrieval augmented generation (RAG) tries to reduce factual errors by identifying information in a knowledge corpus and putting it in the context window of the model. While this approach is well-established for document-structured data, it is non-trivial to adapt it for Knowledge Graphs (KGs), especially for queries that require multi-node/multi-hop reasoning on graphs. We introduce ULTRAG, a general framework for retrieving information from Knowledge Graphs that shifts away from classical RAG. By endowing LLMs with off-the-shelf neural query executing modules, we highlight how readily available language models can achieve state-of-the-art results on Knowledge Graph Question Answering (KGQA) tasks without any retraining of the LLM or executor involved. In our experiments, ULTRAG achieves better performance when compared to state-of-the-art KG-RAG solutions, and it enables language models to interface with Wikidata-scale graphs (116M entities, 1.6B relations) at comparable or lower costs.