Mode-Conditioning Unlocks Superior Test-Time Scaling
This addresses a key bottleneck in test-time scaling for machine learning models, offering a simple remedy to unlock diversity benefits, though it is incremental in building on existing parallel sampling methods.
The paper tackles the problem of diversity collapse in parallel sampling during test-time scaling, where models concentrate on a few modes and produce repeated mistakes. It proposes the mode-conditioning (ModC) framework, which improves scaling across tasks and models, achieving up to a 4x efficiency gain and 10% performance improvements.
Parallel sampling promises substantial gains in test-time scaling, but its effectiveness is sharply limited by diversity collapse, where models concentrate on a few modes and repeated samples produce the same mistakes. We propose the mode-conditioning (ModC) framework, which explicitly allocates test-time compute across reasoning modes using either specialist models or mode-specific prefixes. ModC consistently improves scaling across controlled graph-search tasks and large-scale reasoning benchmarks, spanning model families and sizes from 0.5B to 7B. On OpenThoughts, fine-tuning Qwen2.5-7B with ModC achieves a 4x efficiency gain over standard training while also improving the maximum attainable Pass@k. We further show that gradient clustering enables ModC without explicit mode labels, yielding up to 10% gains on datasets such as NuminaMath. Finally, we show that ModC improves reinforcement learning (RL) and can further boost diversity-inducing RL methods. These results demonstrate that standard training underutilizes the diversity in data, and that ModC provides a simple, effective remedy for unlocking the full benefits of diversity in test-time scaling.