Careful Queries, Credible Results: Teaching RAG Models Advanced Web Search Tools with Reinforcement Learning
This addresses the challenge of unreliable web content for RAG systems, offering an incremental improvement in retrieval accuracy.
The paper tackled the problem of misinformation and underutilization of web tools in Retrieval-Augmented Generation (RAG) systems by proposing WebFilter, a framework that generates source-restricted queries and filters unreliable content, resulting in improved answer quality and retrieval precision that outperforms existing methods on benchmarks.
Retrieval-Augmented Generation (RAG) enhances large language models (LLMs) by integrating up-to-date external knowledge, yet real-world web environments present unique challenges. These limitations manifest as two key challenges: pervasive misinformation in the web environment, which introduces unreliable or misleading content that can degrade retrieval accuracy, and the underutilization of web tools, which, if effectively employed, could enhance query precision and help mitigate this noise, ultimately improving the retrieval results in RAG systems. To address these issues, we propose WebFilter, a novel RAG framework that generates source-restricted queries and filters out unreliable content. This approach combines a retrieval filtering mechanism with a behavior- and outcome-driven reward strategy, optimizing both query formulation and retrieval outcomes. Extensive experiments demonstrate that WebFilter improves answer quality and retrieval precision, outperforming existing RAG methods on both in-domain and out-of-domain benchmarks.