Wider or Deeper? Scaling LLM Inference-Time Compute with Adaptive Branching Tree Search
This work addresses the challenge of improving LLM reasoning capabilities during inference for tasks like coding, though it is incremental as it builds on existing methods like MCTS and repeated sampling.
The paper tackled the problem of scaling inference-time computation for large language models (LLMs) by proposing Adaptive Branching Monte Carlo Tree Search (AB-MCTS), a framework that dynamically balances exploring new candidate responses and exploiting existing ones using external feedback, and it consistently outperformed repeated sampling and standard MCTS on complex coding and engineering tasks.
Recent advances demonstrate that increasing inference-time computation can significantly boost the reasoning capabilities of large language models (LLMs). Although repeated sampling (i.e., generating multiple candidate outputs) is a highly effective strategy, it does not leverage external feedback signals for refinement, which are often available in tasks like coding. In this work, we propose Adaptive Branching Monte Carlo Tree Search (AB-MCTS), a novel inference-time framework that generalizes repeated sampling with principled multi-turn exploration and exploitation. At each node in the search tree, AB-MCTS dynamically decides whether to "go wider" by expanding new candidate responses or "go deeper" by revisiting existing ones based on external feedback signals. We evaluate our method on complex coding and engineering tasks using frontier models. Empirical results show that AB-MCTS consistently outperforms both repeated sampling and standard MCTS, underscoring the importance of combining the response diversity of LLMs with multi-turn solution refinement for effective inference-time scaling. Code is available at https://github.com/SakanaAI/treequest .