GraphRAFT: Retrieval Augmented Fine-Tuning for Knowledge Graphs on Graph Databases
It addresses a critical bottleneck for organizations using graph databases by providing an efficient, off-the-shelf solution for accurate knowledge graph querying, though it is incremental as it builds on existing GraphRAG methods.
The paper tackles the problem of hallucination in large language models when querying private data on knowledge graphs stored in graph databases, by introducing GraphRAFT, a framework that fine-tunes LLMs to generate correct Cypher queries for retrieval and reasoning, achieving significantly better results than state-of-the-art models across all metrics on challenging Q&A benchmarks.
Large language models have shown remarkable language processing and reasoning ability but are prone to hallucinate when asked about private data. Retrieval-augmented generation (RAG) retrieves relevant data that fit into an LLM's context window and prompts the LLM for an answer. GraphRAG extends this approach to structured Knowledge Graphs (KGs) and questions regarding entities multiple hops away. The majority of recent GraphRAG methods either overlook the retrieval step or have ad hoc retrieval processes that are abstract or inefficient. This prevents them from being adopted when the KGs are stored in graph databases supporting graph query languages. In this work, we present GraphRAFT, a retrieve-and-reason framework that finetunes LLMs to generate provably correct Cypher queries to retrieve high-quality subgraph contexts and produce accurate answers. Our method is the first such solution that can be taken off-the-shelf and used on KGs stored in native graph DBs. Benchmarks suggest that our method is sample-efficient and scales with the availability of training data. Our method achieves significantly better results than all state-of-the-art models across all four standard metrics on two challenging Q&As on large text-attributed KGs.