Group Equivariant Subsampling
This addresses the problem of building more efficient and generalizable neural networks for computer vision tasks by ensuring equivariance to transformations like translations and rotations, though it is incremental as it extends existing equivariant methods to subsampling.
The paper tackled the problem of subsampling operations in CNNs not being translation equivariant, unlike convolutions, by introducing translation and group equivariant subsampling/upsampling layers to construct exact group equivariant CNNs and autoencoders. The result showed empirically that these representations are equivariant to input translations and rotations, leading to improved data efficiency and decomposition in multi-object datasets compared to non-equivariant baselines.
Subsampling is used in convolutional neural networks (CNNs) in the form of pooling or strided convolutions, to reduce the spatial dimensions of feature maps and to allow the receptive fields to grow exponentially with depth. However, it is known that such subsampling operations are not translation equivariant, unlike convolutions that are translation equivariant. Here, we first introduce translation equivariant subsampling/upsampling layers that can be used to construct exact translation equivariant CNNs. We then generalise these layers beyond translations to general groups, thus proposing group equivariant subsampling/upsampling. We use these layers to construct group equivariant autoencoders (GAEs) that allow us to learn low-dimensional equivariant representations. We empirically verify on images that the representations are indeed equivariant to input translations and rotations, and thus generalise well to unseen positions and orientations. We further use GAEs in models that learn object-centric representations on multi-object datasets, and show improved data efficiency and decomposition compared to non-equivariant baselines.