Retrieval-Augmented Generation for Natural Language Processing: A Survey
It provides a comprehensive overview for researchers and practitioners in NLP, but is incremental as it synthesizes existing work rather than introducing new findings.
This survey addresses the limitations of large language models (LLMs), such as hallucination and knowledge gaps, by reviewing retrieval-augmented generation (RAG) techniques that use external knowledge databases to augment LLMs, covering methods, applications, and future directions.
Large language models (LLMs) have demonstrated great success in various fields, benefiting from their huge amount of parameters that store knowledge. However, LLMs still suffer from several key issues, such as hallucination problems, knowledge update issues, and lacking domain-specific expertise. The appearance of retrieval-augmented generation (RAG), which leverages an external knowledge database to augment LLMs, makes up those drawbacks of LLMs. This paper reviews all significant techniques of RAG, especially in the retriever and the retrieval fusions. Besides, tutorial codes are provided for implementing the representative techniques in RAG. This paper further discusses the RAG update, including RAG with/without knowledge update. Then, we introduce RAG evaluation and benchmarking, as well as the application of RAG in representative NLP tasks and industrial scenarios. Finally, this paper discusses RAG's future directions and challenges for promoting this field's development.