Encoding Involutory Invariances in Neural Networks
This work addresses the issue of symmetry preservation in neural networks for domains with inherent symmetries, such as image datasets with reflection symmetry, though it appears incremental as it builds on existing symmetry-embedding techniques.
The paper tackled the problem of neural networks not respecting underlying symmetries in data by introducing architectures that embed invariance to involutory linear/affine transformations, and it demonstrated that the proposed model outperforms baseline networks while exactly respecting the symmetry.
In certain situations, neural networks are trained upon data that obey underlying symmetries. However, the predictions do not respect the symmetries exactly unless embedded in the network structure. In this work, we introduce architectures that embed a special kind of symmetry namely, invariance with respect to involutory linear/affine transformations up to parity $p=\pm 1$. We provide rigorous theorems to show that the proposed network ensures such an invariance and present qualitative arguments for a special universal approximation theorem. An adaption of our techniques to CNN tasks for datasets with inherent horizontal/vertical reflection symmetry is demonstrated. Extensive experiments indicate that the proposed model outperforms baseline feed-forward and physics-informed neural networks while identically respecting the underlying symmetry.