Equivariant and Steerable Neural Networks: A review with special emphasis on the symmetric group
This is an incremental review that synthesizes existing concepts for researchers in machine learning and symmetry-based neural networks.
The paper reviews the architecture of group equivariant and steerable neural networks, which generalize convolutional neural networks by incorporating symmetry group invariances, and applies this formalism to the symmetric group to provide new details on representations and capsules not previously documented.
Convolutional neural networks revolutionized computer vision and natrual language processing. Their efficiency, as compared to fully connected neural networks, has its origin in the architecture, where convolutions reflect the translation invariance in space and time in pattern or speech recognition tasks. Recently, Cohen and Welling have put this in the broader perspective of invariance under symmetry groups, which leads to the concept of group equivaiant neural networks and more generally steerable neural networks. In this article, we review the architecture of such networks including equivariant layers and filter banks, activation with capsules and group pooling. We apply this formalism to the symmetric group, for which we work out a number of details on representations and capsules that are not found in the literature.