Recursion of Thought: A Divide-and-Conquer Approach to Multi-Context Reasoning with Language Models
This addresses a key bottleneck in scaling reasoning tasks for language model users, offering a novel alternative to increasing context size.
The paper tackles the problem of language models exceeding context limits in multi-step reasoning by proposing Recursion of Thought (RoT), a divide-and-conquer framework that enables models to handle problems with solutions of hundreds of thousands of tokens, significantly improving inference capability.
Generating intermediate steps, or Chain of Thought (CoT), is an effective way to significantly improve language models' (LM) multi-step reasoning capability. However, the CoT lengths can grow rapidly with the problem complexity, easily exceeding the maximum context size. Instead of increasing the context limit, which has already been heavily investigated, we explore an orthogonal direction: making LMs divide a problem into multiple contexts. We propose a new inference framework, called Recursion of Thought (RoT), which introduces several special tokens that the models can output to trigger context-related operations. Extensive experiments with multiple architectures including GPT-3 show that RoT dramatically improves LMs' inference capability to solve problems, whose solution consists of hundreds of thousands of tokens.