Convolutional layers are equivariant to discrete shifts but not continuous translations
This is an incremental clarification for researchers in machine learning to improve terminology and understanding of CNN properties.
The paper clarifies that convolutional layers in CNNs are equivariant to discrete pixel shifts but not to continuous translations, addressing a common misconception among non-experts.
The purpose of this short and simple note is to clarify a common misconception about convolutional neural networks (CNNs). CNNs are made up of convolutional layers which are shift equivariant due to weight sharing. However, convolutional layers are not translation equivariant, even when boundary effects are ignored and when pooling and subsampling are absent. This is because shift equivariance is a discrete symmetry while translation equivariance is a continuous symmetry. This fact is well known among researchers in equivariant machine learning, but is usually overlooked among non-experts. To minimize confusion, we suggest using the term `shift equivariance' to refer to discrete shifts in pixels and `translation equivariance' to refer to continuous translations.