Beyond Reward Engineering: A Data Recipe for Long-Context Reinforcement Learning
For practitioners of long-context LLM reasoning, this work provides a simple, effective data recipe that outperforms prior RL training sets without complex reward engineering.
The authors propose a data-centric recipe for long-context reinforcement learning, using eight curated datasets (~14K examples) across retrieval, multi-evidence synthesis, and reasoning tasks. With a minimal outcome-based GRPO setup, they achieve average gains of +7.2, +3.2, and +6.4 points on three model sizes across seven long-context benchmarks, and transfer improvements to agentic tasks (GAIA +4.8, BrowseComp +7.0).
Long-context reasoning is an essential capability for large language models, particularly when they are deployed as autonomous agents that must reason over lengthy trajectories. Reinforcement learning (RL) has recently emerged as a dominant paradigm for improving this ability, yet existing work largely focuses on reward engineering while diverse training data remains scarce. We revisit this problem from a data-centric perspective and show that a simple yet effective data recipe alone, paired with a minimal outcome-based GRPO setup, suffices to substantially improve long-context reasoning. Our recipe targets three complementary task families -- retrieval, multi-evidence synthesis, and reasoning -- for which we construct and curate eight datasets totaling ~14K examples. Experiments on three models (Qwen3-4B/8B/30B-A3B) yield average gains of +7.2/+3.2/+6.4 points across seven long-context benchmarks, surpassing prior RL training sets. We further demonstrate that these gains transfer to agentic tasks, where continuing RL training on an agent-tuned model with our data recipe improves GAIA by +4.8 and BrowseComp by +7.0 points. We will release our datasets to facilitate future research.