AutoSearch: Adaptive Search Depth for Efficient Agentic RAG via Reinforcement Learning
For developers of agentic RAG systems, this work addresses the bottleneck of balancing search depth and cost, offering an adaptive solution that improves efficiency without sacrificing accuracy.
Agentic RAG systems often perform redundant search steps, increasing cost and latency. AutoSearch uses reinforcement learning to dynamically determine the minimal sufficient search depth, achieving a superior accuracy-efficiency trade-off and reducing over-searching while maintaining search quality.
Agentic retrieval-augmented generation (RAG) systems enable large language models (LLMs) to solve complex tasks through multi-step interaction with external retrieval tools. However, such multi-step interaction often involves redundant search steps, incurring substantial computational cost and latency. Prior work limits search depth (i.e., the number of search steps) to reduce cost, but this often leads to underexploration of complex questions. To address this, we first investigate how search depth affects accuracy and find a minimal sufficient search depth that defines an accuracy-efficiency trade-off, jointly determined by question complexity and the agent's capability. Furthermore, we propose AutoSearch, a reinforcement learning (RL) framework that evaluates each search step via self-generated intermediate answers. By a self-answering mechanism, AutoSearch identifies the minimal sufficient search depth and promotes efficient search by rewarding its attainment while penalizing over-searching. In addition, reward mechanisms are introduced to stabilize search behavior and improve answer quality on complex questions. Extensive experiments on multiple benchmarks show that AutoSearch achieves a superior accuracy-efficiency trade-off, alleviating over-searching while preserving search quality.