Learning Mathematical Rules with Large Language Models
This addresses the challenge of improving mathematical reasoning in AI systems, but it is incremental as it builds on existing fine-tuning methods with synthetic data.
The paper tackles the problem of teaching large language models to learn and apply mathematical rules like distributivity and equation simplification, showing through fine-tuning on synthetic data that the models can generalize and reuse these rules in word problems to some extent.
In this paper, we study the ability of large language models to learn specific mathematical rules such as distributivity or simplifying equations. We present an empirical analysis of their ability to generalize these rules, as well as to reuse them in the context of word problems. For this purpose, we provide a rigorous methodology to build synthetic data incorporating such rules, and perform fine-tuning of large language models on such data. Our experiments show that our model can learn and generalize these rules to some extent, as well as suitably reuse them in the context of word problems.