Improving Compositional Generalization in Math Word Problem Solving
This work addresses the limited discussion on compositional generalization in math word problems, which is crucial for enhancing AI's problem-solving skills in educational and reasoning domains, though it appears incremental by building on existing methods with data augmentation.
The authors tackled the problem of compositional generalization in math word problem solving by introducing a data splitting method and an iterative data augmentation approach, which significantly improved the performance of general methods on evaluated datasets, though specific numerical gains were not detailed in the abstract.
Compositional generalization refers to a model's capability to generalize to newly composed input data based on the data components observed during training. It has triggered a series of compositional generalization analysis on different tasks as generalization is an important aspect of language and problem solving skills. However, the similar discussion on math word problems (MWPs) is limited. In this manuscript, we study compositional generalization in MWP solving. Specifically, we first introduce a data splitting method to create compositional splits from existing MWP datasets. Meanwhile, we synthesize data to isolate the effect of compositions. To improve the compositional generalization in MWP solving, we propose an iterative data augmentation method that includes diverse compositional variation into training data and could collaborate with MWP methods. During the evaluation, we examine a set of methods and find all of them encounter severe performance loss on the evaluated datasets. We also find our data augmentation method could significantly improve the compositional generalization of general MWP methods. Code is available at https://github.com/demoleiwang/CGMWP.