Guaranteeing Knowledge Integration with Joint Decoding for Retrieval-Augmented Generation
This addresses a critical problem for users of RAG systems by improving knowledge integration, though it is incremental as it builds on existing RAG methods.
The paper tackles the integration bottleneck in Retrieval-Augmented Generation (RAG), where LLMs often fail to effectively use retrieved documents due to conflicts with internal knowledge, by introducing GuarantRAG, a framework that decouples reasoning from evidence integration and uses joint decoding, resulting in up to 12.1% accuracy improvement and 16.3% reduction in hallucinations on QA benchmarks.
Retrieval-Augmented Generation (RAG) significantly enhances Large Language Models (LLMs) by providing access to external knowledge. However, current research primarily focuses on retrieval quality, often overlooking the critical ''integration bottleneck'': even when relevant documents are retrieved, LLMs frequently fail to utilize them effectively due to conflicts with their internal parametric knowledge. In this paper, we argue that implicitly resolving this conflict in a single generation pass is suboptimal. We introduce GuarantRAG, a framework that explicitly decouples reasoning from evidence integration. First, we generate an ''Inner-Answer'' based solely on parametric knowledge to capture the model's reasoning flow. Second, to guarantee faithful evidence extraction, we generate a ''Refer-Answer'' using a novel Contrastive DPO objective. This objective treats the parametric Inner-Answer as a negative constraint and the retrieved documents as positive ground truth, forcing the model to suppress internal hallucinations in favor of external evidence during this phase. Finally, rather than naive concatenation or using the DPO trained model directly, we propose a joint decoding mechanism that dynamically fuses the logical coherence of the Inner-Answer with the factual precision of the Refer-Answer at the token level. Experiments on five QA benchmarks demonstrate that GuarantRAG improves accuracy by up to 12.1% and reduces hallucinations by 16.3% compared to standard and dynamic RAG baselines.