From Implicit Exploration to Structured Reasoning: Leveraging Guideline and Refinement for LLMs
This addresses the issue of unreliable reasoning paths in LLMs for general-purpose AI tasks, representing an incremental improvement over existing methods.
The paper tackles the problem of unstable and unguided reasoning in large language models by proposing a framework that uses structured guidelines and stepwise refinement, resulting in consistent performance improvements across multiple benchmarks like BBH, GSM8K, and MATH-500.
Large language models (LLMs) have advanced general-purpose reasoning, showing strong performance across diverse tasks. However, existing methods often rely on implicit exploration, where the model follows stochastic and unguided reasoning paths-like walking without a map. This leads to unstable reasoning paths, lack of error correction, and limited learning from past experience. To address these issues, we propose a framework that shifts from implicit exploration to structured reasoning through guideline and refinement. First, we extract structured reasoning patterns from successful trajectories and reflective signals from failures. During inference, the model follows these guidelines step-by-step, with refinement applied after each step to correct errors and stabilize the reasoning process. Experiments on BBH and four additional benchmarks (GSM8K, MATH-500, MBPP, HumanEval) show that our method consistently outperforms strong baselines across diverse reasoning tasks. Structured reasoning with stepwise execution and refinement improves stability and generalization, while guidelines transfer well across domains and flexibly support cross-model collaboration, matching or surpassing supervised fine-tuning in effectiveness and scalability.