REL: Working out is all you need
This work addresses the problem of enhancing reasoning in language models for AI researchers and practitioners, but it is incremental as it builds on existing methods like Chain-of-Thought and focuses on data generation.
The paper tackles the performance gap in reasoning capabilities between advanced models like OpenAI's O1 and other state-of-the-art LLMs by hypothesizing it's due to limited high-quality reasoning process data, and demonstrates that constructing a specialized dataset of explicit problem-solving workflows elicits substantially improved planning capabilities from existing models.
Recent developments, particularly OpenAI's O1 model, have demonstrated the remarkable potential of Large Language Models (LLMs) for complex reasoning tasks. Through analysis of O1's outputs and provided sample Chain-of-Thought (CoT) demonstrations, we observe that it approaches problem-solving in a distinctly human-like manner, systematically brainstorming ideas, testing hypotheses, verifying results, and planning comprehensive solutions. These sophisticated reasoning capabilities remain notably absent in other state-of-the-art language models. In this paper, we hypothesize that this performance gap stems from the limited availability of high-quality reasoning process data in current training sets. We demonstrate that by constructing a specialized dataset focused on explicit problem-solving workflows ("worked solutions"), we can elicit substantially improved planning capabilities from existing models. Additionally, we propose the Reasoning Enhancement Loop (REL), a method for generating synthetic worked solutions.