RaFe: Ranking Feedback Improves Query Rewriting for RAG
This addresses a generalization issue in query rewriting for RAG, though it appears incremental as it builds on existing methods with a new feedback mechanism.
The paper tackles the problem of training query rewriting models for RAG systems without requiring costly annotations or predesigned rewards, and shows that their framework achieves better performance than baselines.
As Large Language Models (LLMs) and Retrieval Augmentation Generation (RAG) techniques have evolved, query rewriting has been widely incorporated into the RAG system for downstream tasks like open-domain QA. Many works have attempted to utilize small models with reinforcement learning rather than costly LLMs to improve query rewriting. However, current methods require annotations (e.g., labeled relevant documents or downstream answers) or predesigned rewards for feedback, which lack generalization, and fail to utilize signals tailored for query rewriting. In this paper, we propose ours, a framework for training query rewriting models free of annotations. By leveraging a publicly available reranker, ours~provides feedback aligned well with the rewriting objectives. Experimental results demonstrate that ours~can obtain better performance than baselines.