Thinking Long, but Short: Stable Sequential Test-Time Scaling for Large Reasoning Models
This addresses a key limitation in training-free methods for enhancing large reasoning models, offering a more stable and efficient approach for researchers and practitioners.
The paper tackles the problem of accuracy degradation and instability in sequential test-time scaling for large reasoning models by introducing Min-Seek, a method that stabilizes accuracy across a wide range of induced thoughts and removes the need for reasoning length fine-tuning, achieving significant accuracy improvements on various reasoning tasks with linear computational complexity.
Sequential test-time scaling is a promising training-free method to improve large reasoning model accuracy, but as currently implemented, significant limitations have been observed. Inducing models to think for longer can increase their accuracy, but as the length of reasoning is further extended, it has also been shown to result in accuracy degradation and model instability. This work presents a novel sequential test-time scaling method, Min-Seek, which improves model accuracy significantly over a wide range of induced thoughts, stabilizing the accuracy of sequential scaling, and removing the need for reasoning length fine-tuning. Beyond improving model accuracy over a variety of reasoning tasks, our method is inherently efficient, as only the KV pairs of one additional induced thought are kept in the KV cache during reasoning. With a custom KV cache which stores keys without position embeddings, by dynamically encoding them contiguously before each new generated thought, our method can continue to reason well beyond a model's maximum context length, and under mild conditions has linear computational complexity.