Do Not Waste Your Rollouts: Recycling Search Experience for Efficient Test-Time Scaling
This addresses inefficiency in test-time scaling for AI reasoning tasks, offering a training-free method to reduce computational waste, though it is incremental as it builds on existing search strategies.
The paper tackles the problem of computational redundancy in test-time scaling for large language models by proposing Recycling Search Experience (RSE), a strategy that reuses intermediate search insights to avoid redundant derivations and dead ends, achieving state-of-the-art scaling efficiency on benchmarks like HMMT24 and IMO-Bench.
Test-Time Scaling enhances the reasoning capabilities of Large Language Models by allocating additional inference compute to broaden the exploration of the solution space. However, existing search strategies typically treat rollouts as disposable samples, where valuable intermediate insights are effectively discarded after each trial. This systemic memorylessness leads to massive computational redundancy, as models repeatedly re-derive discovered conclusions and revisit known dead ends across extensive attempts. To bridge this gap, we propose \textbf{Recycling Search Experience (RSE)}, a self-guided, training-free strategy that turns test-time search from a series of isolated trials into a cumulative process. By actively distilling raw trajectories into a shared experience bank, RSE enables positive recycling of intermediate conclusions to shortcut redundant derivations and negative recycling of failure patterns to prune encountered dead ends. Theoretically, we provide an analysis that formalizes the efficiency gains of RSE, validating its advantage over independent sampling in solving complex reasoning tasks. Empirically, extensive experiments on HMMT24, HMMT25, IMO-Bench, and HLE show that RSE consistently outperforms strong baselines with comparable computational cost, achieving state-of-the-art scaling efficiency.