DynScaling: Efficient Verifier-free Inference Scaling via Dynamic and Integrated Sampling
This work addresses practical computational constraints in LLM inference scaling, offering a verifier-free solution that is incremental but impactful for deployment efficiency.
The paper tackles the inefficiency and reliance on external verifiers in inference-time scaling for large language models by proposing DynScaling, which integrates parallel-sequential sampling and dynamic budget allocation, resulting in improved performance and reduced computational cost compared to existing baselines.
Inference-time scaling has proven effective in boosting large language model (LLM) performance through increased test-time computation. Yet, its practical application is often hindered by reliance on external verifiers or a lack of optimization for realistic computational constraints. We propose DynScaling, which addresses these limitations through two primary innovations: an integrated parallel-sequential sampling strategy and a bandit-based dynamic budget allocation framework. The integrated sampling strategy unifies parallel and sequential sampling by constructing synthetic sequential reasoning chains from initially independent parallel responses, promoting diverse and coherent reasoning trajectories. The dynamic budget allocation framework formulates the allocation of computational resources as a multi-armed bandit problem, adaptively distributing the inference budget across queries based on the uncertainty of previously sampled responses, thereby maximizing computational efficiency. By combining these components, DynScaling effectively improves LLM performance under practical resource constraints without the need for external verifiers. Experimental results demonstrate that DynScaling consistently surpasses existing verifier-free inference scaling baselines in both task performance and computational cost.