Optimizing Chain-of-Thought Reasoning: Tackling Arranging Bottleneck via Plan Augmentation
This work addresses a specific bottleneck in multi-step reasoning for large language models, offering an incremental improvement over existing fine-tuning methods.
The paper tackles the bottleneck in Chain-of-Thought reasoning by identifying arranging as the key issue and proposes a plan-based method to guide models, achieving improved performance on GSM8k and ToolBench benchmarks with better generalization in long-distance reasoning.
Multi-step reasoning ability of large language models is crucial in tasks such as math and tool utilization. Current researches predominantly focus on enhancing model performance in these multi-step reasoning tasks through fine-tuning with Chain-of-Thought (CoT) steps, yet these methods tend to be heuristic, without exploring nor resolving the bottleneck. In this study, we subdivide CoT reasoning into two parts: arranging and executing, and identify that the bottleneck of models mainly lies in arranging rather than executing. Based on this finding, we propose a plan-based training and reasoning method that guides models to generate arranging steps through abstract plans. We experiment on both math (GSM8k) and tool utilization (ToolBench) benchmarks. Results show that compared to fine-tuning directly with CoT data, our approach achieves a better performance on alleviating arranging bottleneck, particularly excelling in long-distance reasoning generalization.