Scaling Reasoning Tokens via RL and Parallel Thinking: Evidence From Competitive Programming
This addresses the challenge of efficient reasoning in AI for competitive programming, though it is incremental as it builds on existing methods like RL and parallel processing.
The paper tackled scaling reasoning token budgets for competitive programming by combining reinforcement learning training with a test-time parallel thinking pipeline, achieving performance matching an oracle pass@16 at pass@1 using 7.6 million tokens per problem and surpassing GPT-5-high on 456 hard problems.
We study how to scale reasoning token budgets for competitive programming through two complementary approaches: training-time reinforcement learning (RL) and test-time parallel thinking. During RL training, we observe an approximately log-linear relationship between validation accuracy and the average number of generated reasoning tokens over successive checkpoints, and show two ways to shift this training trajectory: verification RL warmup raises the starting point, while randomized clipping produces a steeper trend in the observed regime. As scaling single-generation reasoning during RL quickly becomes expensive under full attention, we introduce a multi-round parallel thinking pipeline that distributes the token budget across threads and rounds of generation, verification, and refinement. We train the model end-to-end on this pipeline to match the training objective to the test-time structure. Starting from Seed-OSS-36B, the full system with 16 threads and 16 rounds per thread matches the underlying RL model's oracle pass@16 at pass@1 using 7.6 million tokens per problem on average, and surpasses GPT-5-high on 456 hard competitive programming problems from AetherCode.