Diffeomorphism-Equivariant Neural Networks
This work addresses the problem of improving efficiency and data demands in deep learning for researchers and practitioners by enabling equivariance to complex transformations, though it is incremental as it builds on existing equivariance and registration methods.
The paper tackled the challenge of extending equivariance in neural networks to infinite-dimensional groups like diffeomorphisms, proposing an energy-based canonicalisation method that achieves approximate equivariance and generalizes to unseen transformations without extensive data augmentation or retraining.
Incorporating group symmetries via equivariance into neural networks has emerged as a robust approach for overcoming the efficiency and data demands of modern deep learning. While most existing approaches, such as group convolutions and averaging-based methods, focus on compact, finite, or low-dimensional groups with linear actions, this work explores how equivariance can be extended to infinite-dimensional groups. We propose a strategy designed to induce diffeomorphism equivariance in pre-trained neural networks via energy-based canonicalisation. Formulating equivariance as an optimisation problem allows us to access the rich toolbox of already established differentiable image registration methods. Empirical results on segmentation and classification tasks confirm that our approach achieves approximate equivariance and generalises to unseen transformations without relying on extensive data augmentation or retraining.