SBSC: Step-By-Step Coding for Improving Mathematical Olympiad Performance
This work addresses the challenge of improving mathematical reasoning for AI systems, particularly in competitive settings, but it is incremental as it builds on existing program generation methods.
The paper tackles the problem of solving Olympiad-level math problems by proposing Step-by-Step Coding (SBSC), a multi-turn framework that enables Large Language Models to generate sequences of programs, resulting in absolute performance gains of up to 12.6% over existing state-of-the-art methods on benchmarks like AMC12, AIME, and MathOdyssey.
We propose Step-by-Step Coding (SBSC): a multi-turn math reasoning framework that enables Large Language Models (LLMs) to generate sequence of programs for solving Olympiad level math problems. At each step/turn, by leveraging the code execution outputs and programs of previous steps, the model generates the next sub-task and the corresponding program to solve it. This way, SBSC, sequentially navigates to reach the final answer. SBSC allows more granular, flexible and precise approach to problem-solving compared to existing methods. Extensive experiments highlight the effectiveness of SBSC in tackling competition and Olympiad-level math problems. For Claude-3.5-Sonnet, we observe SBSC (greedy decoding) surpasses existing state-of-the-art (SOTA) program generation based reasoning strategies by absolute 10.7% on AMC12, 8% on AIME and 12.6% on MathOdyssey. Given SBSC is multi-turn in nature, we also benchmark SBSC's greedy decoding against self-consistency decoding results of existing SOTA math reasoning strategies and observe performance gain by absolute 6.2% on AMC, 6.7% on AIME and 7.4% on MathOdyssey.