OThink-SRR1: Search, Refine and Reasoning with Reinforced Learning for Large Language Models
This addresses efficiency and accuracy challenges in information-seeking agents for multi-hop question answering, representing a strong incremental improvement over existing dynamic retrieval methods.
The paper tackles the problem of irrelevant noise and high computational costs in retrieval-augmented generation for complex multi-hop questions by proposing OThink-SRR1, a framework that uses reinforcement learning to iteratively search, refine, and reason, achieving superior accuracy on four benchmarks while using fewer retrieval steps and tokens.
Retrieval-Augmented Generation (RAG) expands the knowledge of Large Language Models (LLMs), yet current static retrieval methods struggle with complex, multi-hop problems. While recent dynamic retrieval strategies offer improvements, they face two key challenges: 1) irrelevant retrieved noise can misdirect the reasoning process, and 2) processing full documents incurs prohibitive computational and latency costs. To address these issues, we propose OThink-SRR1, a framework that enhances large models with an iterative Search-Refine-Reason process trained via reinforcement learning. Its core Refine stage distills retrieved documents into concise, relevant facts before reasoning. We introduce GRPO-IR, an end-to-end reinforcement learning algorithm that rewards accurate evidence identification while penalizing excessive retrievals, thus training the model to be both focused and efficient. Experiments on four multi-hop QA benchmarks show our approach achieves superior accuracy over strong baselines while using fewer retrieval steps and tokens. This positions OThink-SRR1 as a potent foundational model for information-seeking agents.