Learning the symmetric group: large from small
This addresses the problem of data scarcity and computational cost in applying machine learning to pure mathematics, though it is incremental as it builds on existing transformer methods.
The paper tackles the challenge of training machine learning models on mathematical datasets by proposing a method where models trained on simpler tasks generalize to more complex ones, demonstrating that a transformer trained on permutations in the symmetric group S10 achieves near 100% accuracy when generalized to S25.
Machine learning explorations can make significant inroads into solving difficult problems in pure mathematics. One advantage of this approach is that mathematical datasets do not suffer from noise, but a challenge is the amount of data required to train these models and that this data can be computationally expensive to generate. Key challenges further comprise difficulty in a posteriori interpretation of statistical models and the implementation of deep and abstract mathematical problems. We propose a method for scalable tasks, by which models trained on simpler versions of a task can then generalize to the full task. Specifically, we demonstrate that a transformer neural-network trained on predicting permutations from words formed by general transpositions in the symmetric group $S_{10}$ can generalize to the symmetric group $S_{25}$ with near 100\% accuracy. We also show that $S_{10}$ generalizes to $S_{16}$ with similar performance if we only use adjacent transpositions. We employ identity augmentation as a key tool to manage variable word lengths, and partitioned windows for training on adjacent transpositions. Finally we compare variations of the method used and discuss potential challenges with extending the method to other tasks.