Adversarial Yet Cooperative: Multi-Perspective Reasoning in Retrieved-Augmented Language Models
This addresses a domain-specific problem for improving reasoning in language models, with incremental contributions.
The paper tackles the problem of limited self-correcting reasoning and insufficient training signals in retrieval-augmented language models by proposing an adversarial Reasoner-Verifier framework, which shows effectiveness on multiple benchmarks.
Recent advances in synergizing large reasoning models (LRMs) with retrieval-augmented generation (RAG) have shown promising results, yet two critical challenges remain: (1) reasoning models typically operate from a single, unchallenged perspective, limiting their ability to conduct deep, self-correcting reasoning over external documents, and (2) existing training paradigms rely excessively on outcome-oriented rewards, which provide insufficient signal for shaping the complex, multi-step reasoning process. To address these issues, we propose an Reasoner-Verifier framework named Adversarial Reasoning RAG (ARR). The Reasoner and Verifier engage in reasoning on retrieved evidence and critiquing each other's logic while being guided by process-aware advantage that requires no external scoring model. This reward combines explicit observational signals with internal model uncertainty to jointly optimize reasoning fidelity and verification rigor. Experiments on multiple benchmarks demonstrate the effectiveness of our method.