RAG-Instruct: Boosting LLMs with Diverse Retrieval-Augmented Instructions
This addresses the problem of limited task diversity and scenario coverage in RAG for LLM researchers and practitioners, representing an incremental improvement through data synthesis.
The paper tackles the limitations of current Retrieval-Augmented Generation (RAG) methods by proposing RAG-Instruct, a method for synthesizing diverse RAG instruction data, resulting in a 40K dataset that enhances LLMs' RAG capabilities with strong zero-shot performance and outperforms baselines across tasks.
Retrieval-Augmented Generation (RAG) has emerged as a key paradigm for enhancing large language models (LLMs) by incorporating external knowledge. However, current RAG methods face two limitations: (1) they only cover limited RAG scenarios. (2) They suffer from limited task diversity due to the lack of a general RAG dataset. To address these limitations, we propose RAG-Instruct, a general method for synthesizing diverse and high-quality RAG instruction data based on any source corpus. Our approach leverages (1) five RAG paradigms, which encompass diverse query-document relationships, and (2) instruction simulation, which enhances instruction diversity and quality by utilizing the strengths of existing instruction datasets. Using this method, we construct a 40K instruction dataset from Wikipedia, comprehensively covering diverse RAG scenarios and tasks. Experiments demonstrate that RAG-Instruct effectively enhances LLMs' RAG capabilities, achieving strong zero-shot performance and significantly outperforming various RAG baselines across a diverse set of tasks. RAG-Instruct is publicly available at https://github.com/FreedomIntelligence/RAG-Instruct.