Making Mathematical Reasoning Adaptive
This work addresses robustness and generalization issues in LLMs for mathematical reasoning, which is incremental as it builds on existing methods to enhance logic-based problem-solving.
The paper tackles the problem of spurious reasoning in large language models (LLMs) for mathematical tasks by proposing the AdaR framework, which uses data synthesis and reinforcement learning to penalize superficial logic and encourage adaptive reasoning, resulting in improved robustness and generalization with substantial gains in mathematical reasoning performance.
Mathematical reasoning is a primary indicator of large language models (LLMs) intelligence. However, existing LLMs exhibit failures of robustness and generalization. This paper attributes these deficiencies to spurious reasoning, i.e., producing answers from superficial features. To address this challenge, we propose the AdaR framework to enable adaptive reasoning, wherein models rely on problem-solving logic to produce answers. AdaR synthesizes logically equivalent queries by varying variable values, and trains models with RLVR on these data to penalize spurious logic while encouraging adaptive logic. To improve data quality, we extract the problem-solving logic from the original query and generate the corresponding answer by code execution, then apply a sanity check. Experimental results demonstrate that AdaR improves robustness and generalization, achieving substantial improvement in mathematical reasoning while maintaining high data efficiency. Analysis indicates that data synthesis and RLVR function in a coordinated manner to enable adaptive reasoning in LLMs. Subsequent analyses derive key design insights into the effect of critical factors and the applicability to instruct LLMs. Our project is available at https://github.com/NJUNLP/AdaR.