Zero-shot Graph Reasoning via Retrieval Augmented Framework with LLMs
This addresses graph reasoning problems for AI researchers and practitioners by offering a scalable, accurate, and efficient solution without requiring fine-tuning, though it is incremental in combining existing techniques like RAG and LLMs.
The authors tackled graph reasoning tasks by proposing a training-free method that uses retrieval-augmented generation with LLMs to generate executable code queries, achieving 100% accuracy on most tasks like cycle detection and shortest path computation, and scaling to graphs with up to 10,000 nodes.
We propose a new, training-free method, Graph Reasoning via Retrieval Augmented Framework (GRRAF), that harnesses retrieval-augmented generation (RAG) alongside the code-generation capabilities of large language models (LLMs) to address a wide range of graph reasoning tasks. In GRRAF, the target graph is stored in a graph database, and the LLM is prompted to generate executable code queries that retrieve the necessary information. This approach circumvents the limitations of existing methods that require extensive finetuning or depend on predefined algorithms, and it incorporates an error feedback loop with a time-out mechanism to ensure both correctness and efficiency. Experimental evaluations on the GraphInstruct dataset reveal that GRRAF achieves 100% accuracy on most graph reasoning tasks, including cycle detection, bipartite graph checks, shortest path computation, and maximum flow, while maintaining consistent token costs regardless of graph sizes. Imperfect but still very high performance is observed on subgraph matching. Notably, GRRAF scales effectively to large graphs with up to 10,000 nodes.