LADDER: Self-Improving LLMs Through Recursive Problem Decomposition
This addresses the challenge of enhancing LLM performance without human supervision or scaling, though it is incremental as it builds on self-learning methods.
The paper tackles the problem of improving LLMs' problem-solving capabilities autonomously, achieving results such as increasing Llama 3.2 3B's accuracy from 1% to 82% on undergraduate-level integration problems and enabling Qwen2.5 7B to reach 90% on the MIT Integration Bee, surpassing OpenAI o1.
We introduce LADDER (Learning through Autonomous Difficulty-Driven Example Recursion), a framework which enables Large Language Models to autonomously improve their problem-solving capabilities through self-guided learning by recursively generating and solving progressively simpler variants of complex problems. Unlike prior approaches that require curated datasets or human feedback, LADDER leverages a model's own capabilities to generate easier question variants. We demonstrate LADDER's effectiveness in the subject of mathematical integration, improving Llama 3.2 3B's accuracy from 1% to 82% on undergraduate-level problems and enabling Qwen2.5 7B Deepseek-R1 Distilled to achieve 73% on the MIT Integration Bee qualifying examination. We also introduce TTRL (Test-Time Reinforcement Learning), where we perform reinforcement learning on variants of test problems at inference time. TTRL enables Qwen2.5 7B Deepseek-R1 Distilled to achieve a state-of-the-art score of 90% on the MIT Integration Bee qualifying examination, surpassing OpenAI o1's performance. These results show how self-directed strategic learning can achieve significant capability improvements without relying on architectural scaling or human supervision.