Review of Inference-Time Scaling Strategies: Reasoning, Search and RAG
This is an incremental review paper that organizes existing techniques for improving LLM performance at inference time, targeting researchers and practitioners in AI and NLP.
The paper reviews inference-time scaling strategies for LLMs, addressing the bottleneck of limited high-quality training data by using additional computation at deployment to improve performance on downstream tasks without re-training, categorizing techniques into output-focused (e.g., reasoning, search) and input-focused (e.g., RAG) methods.
The performance gains of LLMs have historically been driven by scaling up model size and training data. However, the rapidly diminishing availability of high-quality training data is introducing a fundamental bottleneck, shifting the focus of research toward inference-time scaling. This paradigm uses additional computation at the time of deployment to substantially improve LLM performance on downstream tasks without costly model re-training. This review systematically surveys the diverse techniques contributing to this new era of inference-time scaling, organizing the rapidly evolving field into two comprehensive perspectives: Output-focused and Input-focused methods. Output-focused techniques encompass complex, multi-step generation strategies, including reasoning (e.g., CoT, ToT, ReAct), various search and decoding methods (e.g., MCTS, beam search), training for long CoT (e.g., RLVR, GRPO), and model ensemble methods. Input-focused techniques are primarily categorized by few-shot and RAG, with RAG as the central focus. The RAG section is further detailed through a structured examination of query expansion, data, retrieval and reranker, LLM generation methods, and multi-modal RAG.