DeepNote: Note-Centric Deep Retrieval-Augmented Generation
This addresses the issue of incomplete knowledge utilization in RAG for QA, offering a novel method that is incremental but with strong specific improvements.
The paper tackles the problem of factual errors and hallucinations in LLMs for QA by introducing DeepNote, an adaptive RAG framework that uses notes to refine and accumulate knowledge, achieving performance gains of +10.2% to +20.1% over baselines.
Retrieval-Augmented Generation (RAG) mitigates factual errors and hallucinations in Large Language Models (LLMs) for question-answering (QA) by incorporating external knowledge. However, existing adaptive RAG methods rely on LLMs to predict retrieval timing and directly use retrieved information for generation, often failing to reflect real information needs and fully leverage retrieved knowledge. We develop DeepNote, an adaptive RAG framework that achieves in-depth and robust exploration of knowledge sources through note-centric adaptive retrieval. DeepNote employs notes as carriers for refining and accumulating knowledge. During in-depth exploration, it uses these notes to determine retrieval timing, formulate retrieval queries, and iteratively assess knowledge growth, ultimately leveraging the best note for answer generation. Extensive experiments and analyses demonstrate that DeepNote significantly outperforms all baselines (+10.2% to +20.1%) and exhibits the ability to gather knowledge with both high density and quality. Additionally, DPO further improves the performance of DeepNote. The code and data are available at https://github.com/thunlp/DeepNote.