Fast, Slow, and Tool-augmented Thinking for LLMs: A Review
This work addresses the need for more adaptive and efficient reasoning in LLMs for AI researchers and practitioners, but it is incremental as it reviews and categorizes existing approaches rather than introducing new methods.
The paper tackles the problem of adapting LLM reasoning strategies to real-world tasks by proposing a taxonomy based on fast/slow and internal/external boundaries, and surveys existing methods to categorize them for improved adaptability.
Large Language Models (LLMs) have demonstrated remarkable progress in reasoning across diverse domains. However, effective reasoning in real-world tasks requires adapting the reasoning strategy to the demands of the problem, ranging from fast, intuitive responses to deliberate, step-by-step reasoning and tool-augmented thinking. Drawing inspiration from cognitive psychology, we propose a novel taxonomy of LLM reasoning strategies along two knowledge boundaries: a fast/slow boundary separating intuitive from deliberative processes, and an internal/external boundary distinguishing reasoning grounded in the model's parameters from reasoning augmented by external tools. We systematically survey recent work on adaptive reasoning in LLMs and categorize methods based on key decision factors. We conclude by highlighting open challenges and future directions toward more adaptive, efficient, and reliable LLMs.