SIGHT: Reinforcement Learning with Self-Evidence and Information-Gain Diverse Branching for Search Agent
This addresses the issue of 'Tunnel Vision' and error accumulation in search-based reasoning for AI agents, representing a novel method for a known bottleneck rather than a foundational breakthrough.
The paper tackles the problem of high redundancy and low signal-to-noise in multi-turn search for question answering with reinforcement learning, resulting in SIGHT, a framework that significantly outperforms existing approaches in complex reasoning scenarios using fewer search steps.
Reinforcement Learning (RL) has empowered Large Language Models (LLMs) to master autonomous search for complex question answering. However, particularly within multi-turn search scenarios, this interaction introduces a critical challenge: search results often suffer from high redundancy and low signal-to-noise ratios. Consequently, agents easily fall into "Tunnel Vision," where the forced interpretation of early noisy retrievals leads to irreversible error accumulation. To address these challenges, we propose SIGHT, a framework that enhances search-based reasoning through Self-Evidence Support (SES) and Information-Gain Driven Diverse Branching. SIGHT distills search results into high-fidelity evidence via SES and calculates an Information Gain score to pinpoint pivotal states where observations maximally reduce uncertainty. This score guides Dynamic Prompting Interventions - including de-duplication, reflection, or adaptive branching - to spawn new branches with SES. Finally, by integrating SES and correctness rewards via Group Relative Policy Optimization, SIGHT internalizes robust exploration strategies without external verifiers. Experiments on single-hop and multi-hop QA benchmarks demonstrate that SIGHT significantly outperforms existing approaches, particularly in complex reasoning scenarios, using fewer search steps.