Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Models
This work addresses the underexplored area of compute-optimal inference for LLMs, offering insights that could reduce computational costs for problem-solving tasks, though it is incremental as it builds on existing inference methods.
The study tackled the problem of optimizing inference compute for large language models by exploring trade-offs between model size and inference strategies, finding that scaling inference compute with advanced strategies like tree search can be more efficient than scaling model parameters, with the Llemma-7B model outperforming Llemma-34B on the MATH benchmark when paired with a novel tree search algorithm.
While the scaling laws of large language models (LLMs) training have been extensively studied, optimal inference configurations of LLMs remain underexplored. We study inference scaling laws (aka test-time scaling laws) and compute-optimal inference, focusing on the trade-offs between model sizes and generating additional tokens with different inference strategies. As a first step towards understanding and designing compute-optimal inference methods, we studied cost-performance trade-offs for inference strategies such as greedy search, majority voting, best-of-$n$, weighted voting, and two different tree search algorithms, using different model sizes and compute budgets. Our findings suggest that scaling inference compute with inference strategies can be more computationally efficient than scaling model parameters. Additionally, smaller models combined with advanced inference algorithms offer Pareto-optimal trade-offs in cost and performance. For example, the Llemma-7B model, when paired with our novel tree search algorithm, consistently outperforms the Llemma-34B model across all tested inference strategies on the MATH benchmark. We hope these insights contribute to a deeper understanding of inference scaling laws (test-time scaling laws) for LLMs.