Unified Active Retrieval for Retrieval Augmented Generation
This addresses the sub-optimality of always retrieving in RAG for users of AI systems, but it is incremental as it builds on existing active retrieval methods.
The paper tackled the problem of determining when to retrieve in Retrieval-Augmented Generation (RAG) by proposing Unified Active Retrieval (UAR), which uses four orthogonal criteria to improve retrieval timing judgments and downstream task performance, as shown in experiments on four types of user instructions.
In Retrieval-Augmented Generation (RAG), retrieval is not always helpful and applying it to every instruction is sub-optimal. Therefore, determining whether to retrieve is crucial for RAG, which is usually referred to as Active Retrieval. However, existing active retrieval methods face two challenges: 1. They usually rely on a single criterion, which struggles with handling various types of instructions. 2. They depend on specialized and highly differentiated procedures, and thus combining them makes the RAG system more complicated and leads to higher response latency. To address these challenges, we propose Unified Active Retrieval (UAR). UAR contains four orthogonal criteria and casts them into plug-and-play classification tasks, which achieves multifaceted retrieval timing judgements with negligible extra inference cost. We further introduce the Unified Active Retrieval Criteria (UAR-Criteria), designed to process diverse active retrieval scenarios through a standardized procedure. Experiments on four representative types of user instructions show that UAR significantly outperforms existing work on the retrieval timing judgement and the performance of downstream tasks, which shows the effectiveness of UAR and its helpfulness to downstream tasks.