DAGR: Decomposition Augmented Graph Retrieval with LLMs
This addresses the limitation of LLMs in efficiently reasoning over graph-structured data for complex question answering, representing an incremental improvement in retrieval methods.
The paper tackles the problem of LLMs struggling with multi-hop reasoning and factual consistency in knowledge-intensive tasks like complex QA by introducing DAGR, a retrieval method that decomposes complex queries and extracts relevant subgraphs, achieving comparable or superior performance on standard benchmarks with smaller models and fewer LLM calls.
Large Language Models (LLMs) excel at many Natural Language Processing (NLP) tasks, but struggle with multi-hop reasoning and factual consistency, limiting their effectiveness on knowledge-intensive tasks like complex question answering (QA). Linking Knowledge Graphs (KG) and LLMs has shown promising results, but LLMs generally lack the ability to reason efficiently over graph-structured information. To address this challenge, we introduce DAGR, a retrieval method that leverages both complex questions and their decomposition in subquestions to extract relevant, linked textual subgraphs. DAGR first breaks down complex queries, retrieves subgraphs guided by a weighted similarity function over both the original and decomposed queries, and creates a question-specific knowledge graph to guide answer generation. The resulting Graph-RAG pipeline is suited to handle complex multi-hop questions and effectively reason over graph-structured data. We evaluate DAGR on standard multi-hop QA benchmarks and show that it achieves comparable or superior performance to competitive existing methods, using smaller models and fewer LLM calls.