On Symmetries in Convolutional Weights
This provides insight into inherent inductive biases in CNNs, which could inform architecture design for researchers.
The paper investigates why mean convolutional kernels in internal layers of CNNs tend to be symmetric rather than directional, finding this symmetry correlates with shift and flip consistency properties.
We explore the symmetry of the mean k x k weight kernel in each layer of various convolutional neural networks. Unlike individual neurons, the mean kernels in internal layers tend to be symmetric about their centers instead of favoring specific directions. We investigate why this symmetry emerges in various datasets and models, and how it is impacted by certain architectural choices. We show how symmetry correlates with desirable properties such as shift and flip consistency, and might constitute an inherent inductive bias in convolutional neural networks.