Optimizing Retrieval for RAG via Reinforced Contrastive Learning
This addresses the problem of optimizing retrieval for AI systems in RAG, where relevance is hard to predefine, offering a practical solution with incremental gains over existing methods.
The paper tackles the challenge of defining relevance in retrieval-augmented generation (RAG) systems by proposing R3, a retrieval framework that uses reinforced contrastive learning to dynamically optimize relevance, resulting in a 5.2% improvement over the original retriever and surpassing state-of-the-art retrievers by 4.9%.
As retrieval-augmented generation (RAG) becomes increasingly widespread, the role of information retrieval (IR) is shifting from retrieving information for human users to retrieving contextual knowledge for artificial intelligence (AI) systems, where relevance becomes difficult to define or annotate beforehand. To address this challenge, we propose R3, a Retrieval framework optimized for RAG through trialand-feedback Reinforced contrastive learning. Unlike prior approaches that rely on annotated or synthetic data for supervised fine-tuning, R3 enables the retriever to dynamically explore and optimize relevance within the RAG environment. During training, the retrieved results interact with the environment to produce contrastive signals that automatically guide the retriever's self-improvement. Extensive experiments across diverse tasks demonstrate that R3 improves RAG performance by 5.2% over the original retriever and surpasses state-of-the-art retrievers by 4.9%, while achieving comparable results to LLM-augmented retrieval and RAG systems built on post-trained or instruction-tuned LLMs. It is both efficient and practical, requiring only 4 GPUs and completing training within a single day.