Equivariance versus Augmentation for Spherical Images
This work addresses the trade-offs between architectural equivariance and data augmentation for spherical image processing, providing practical insights for researchers and practitioners in computer vision.
The study compared rotational equivariant CNNs (S2CNNs) with standard CNNs using data augmentation on spherical images, finding that for rotationally invariant classification tasks, standard CNNs with heavy augmentation matched equivariant performance, while for equivariant segmentation tasks, equivariant networks outperformed with fewer parameters.
We analyze the role of rotational equivariance in convolutional neural networks (CNNs) applied to spherical images. We compare the performance of the group equivariant networks known as S2CNNs and standard non-equivariant CNNs trained with an increasing amount of data augmentation. The chosen architectures can be considered baseline references for the respective design paradigms. Our models are trained and evaluated on single or multiple items from the MNIST or FashionMNIST dataset projected onto the sphere. For the task of image classification, which is inherently rotationally invariant, we find that by considerably increasing the amount of data augmentation and the size of the networks, it is possible for the standard CNNs to reach at least the same performance as the equivariant network. In contrast, for the inherently equivariant task of semantic segmentation, the non-equivariant networks are consistently outperformed by the equivariant networks with significantly fewer parameters. We also analyze and compare the inference latency and training times of the different networks, enabling detailed tradeoff considerations between equivariant architectures and data augmentation for practical problems. The equivariant spherical networks used in the experiments are available at https://github.com/JanEGerken/sem_seg_s2cnn .