RM-PoT: Reformulating Mathematical Problems and Solving via Program of Thoughts
This addresses robustness issues in LLMs for mathematical reasoning, though it is incremental as it builds on existing step-by-step reasoning methods.
The paper tackles the vulnerability of large language models to surface-level variations in mathematical problems, which impacts solve rates, by proposing RM-PoT, a framework that reformulates problems and uses code-aided reasoning, achieving improved performance on numerical reasoning tasks.
Recently, substantial advancements have been made in training language models to carry out step-by-step reasoning for solving intricate numerical reasoning tasks. Beyond the methods used to solve these problems, the structure and formulation of the problems themselves also play a crucial role in determining the performance of large language models. We observe that even small changes in the surface form of mathematical problems can have a profound impact on both the answer distribution and solve rate. This highlights the vulnerability of LLMs to surface-level variations, revealing its limited robustness when reasoning through complex problems. In this paper, we propose RM-PoT, a three-stage framework that integrates problem reformulation (RM), code-aided reasoning (PoT), and domain-aware few-shot learning to address these limitations. Our approach first reformulates the input problem into diverse surface forms to reduce structural bias, then retrieves five semantically aligned examples from a pre-constructed domain-specific question bank to provide contextual guidance, and finally generates executable Python code for precise computation.