A Group Theoretic Analysis of the Symmetries Underlying Base Addition and Their Learnability by Neural Networks
This addresses a challenge in designing neural networks for cognitive modeling and AI by providing insights into symmetry learning, but it is incremental as it builds on existing group theory and neural network methods.
The paper tackled the problem of enabling neural networks to learn symmetry functions for radical generalization, using base addition as a case study, and found that simple networks can achieve this with specific input formats and carry functions, with learnability correlated to carry function structure.
A major challenge in the use of neural networks both for modeling human cognitive function and for artificial intelligence is the design of systems with the capacity to efficiently learn functions that support radical generalization. At the roots of this is the capacity to discover and implement symmetry functions. In this paper, we investigate a paradigmatic example of radical generalization through the use of symmetry: base addition. We present a group theoretic analysis of base addition, a fundamental and defining characteristic of which is the carry function -- the transfer of the remainder, when a sum exceeds the base modulus, to the next significant place. Our analysis exposes a range of alternative carry functions for a given base, and we introduce quantitative measures to characterize these. We then exploit differences in carry functions to probe the inductive biases of neural networks in symmetry learning, by training neural networks to carry out base addition using different carries, and comparing efficacy and rate of learning as a function of their structure. We find that even simple neural networks can achieve radical generalization with the right input format and carry function, and that learnability is closely correlated with carry function structure. We then discuss the relevance this has for cognitive science and machine learning.