Mitigating Strategy-Selection Bias in Reasoning for More Effective Test-Time Scaling
This addresses a critical bottleneck in improving reasoning performance for large language models, though it is incremental as it builds on existing test-time scaling methods.
The paper tackles the problem of selection bias in reasoning strategies during test-time scaling for large language models, which limits exploration of the solution space, and introduces TTS-Uniform to mitigate this bias, resulting in significant performance improvements across multiple LLMs and benchmarks.
Test-time scaling (TTS) has been shown to improve the performance of large language models (LLMs) by sampling and aggregating diverse reasoning paths. However, existing research has overlooked a critical issue: selection bias of reasoning strategies during scaling. Specifically, when generating reasoning processes, LLMs tend to follow certain strategies (e.g., algebraic solutions for math problems) while neglecting other valid alternatives (e.g., geometric solutions), resulting in insufficient exploration of the solution space. To further understand the impact of this bias, we present a theoretical analysis that reveals when it undermines the effectiveness of test-time scaling. Motivated by this theoretical insight, we introduce TTS-Uniform, a framework designed to mitigate the selection bias of reasoning strategies. It (i) identifies potential strategies, (ii) uniformly allocates the sampling budget across them, and (iii) filters out unstable strategies prior to aggregation. Experimental results show that TTS-Uniform significantly enhances scaling effectiveness across multiple mainstream LLMs and benchmark datasets.