Contextual Compression in Retrieval-Augmented Generation for Large Language Models: A Survey
It provides a comprehensive overview for researchers and practitioners working on improving RAG systems, but is incremental as a survey rather than presenting new experimental results.
This survey explores the evolution of Contextual Compression paradigms in Retrieval-Augmented Generation (RAG) to address limitations like limited context windows and irrelevant information in large language models, outlining current challenges and future research directions.
Large Language Models (LLMs) showcase remarkable abilities, yet they struggle with limitations such as hallucinations, outdated knowledge, opacity, and inexplicable reasoning. To address these challenges, Retrieval-Augmented Generation (RAG) has proven to be a viable solution, leveraging external databases to improve the consistency and coherence of generated content, especially valuable for complex, knowledge-rich tasks, and facilitates continuous improvement by leveraging domain-specific insights. By combining the intrinsic knowledge of LLMs with the vast, dynamic repositories of external databases, RAG achieves a synergistic effect. However, RAG is not without its limitations, including a limited context window, irrelevant information, and the high processing overhead for extensive contextual data. In this comprehensive work, we explore the evolution of Contextual Compression paradigms, providing an in-depth examination of the field. Finally, we outline the current challenges and suggest potential research and development directions, paving the way for future advancements in this area.