Evidence-State Rewards for Long-Context Reasoning
For researchers working on long-context reasoning in LLMs, Maven provides a novel RL approach that optimizes evidence navigation rather than one-shot extraction, addressing the lack of intermediate feedback in existing methods.
Maven, a reinforcement learning framework with editable evidence memory, improves long-context reasoning by rewarding intermediate evidence-state transitions. It outperforms outcome-only RL and evidence-identification baselines on LongBench v2, LongReason, and RULER, producing more sufficient evidence sets and lower distractor retention.
Long-context reasoning requires models to locate, revise, and synthesize evidence distributed across lengthy inputs. Existing long-context RL methods usually reward final answers or static evidence extraction, offering little feedback on how intermediate actions change the model's evidence state. We propose Maven, a reinforcement learning framework with an editable evidence memory. Maven defines an answer-conditioned evidence-state value and rewards action-level state transitions: add actions are credited by marginal gain and hindsight contribution, link actions by evidence synergy, and drop actions by improved answer support after removing misleading evidence. These rewards are assigned to the corresponding action spans in GRPO. Across Llama and Qwen models on LongBench v2, LongReason, and RULER, Maven outperforms outcome-only RL and evidence-identification baselines, producing more sufficient evidence sets and lower distractor retention. Our results show that long-context RL benefits from optimizing stateful evidence navigation rather than one-shot evidence extraction.