Universal Equivariant Multilayer Perceptrons
This provides a theoretical foundation for equivariant networks, which are widely used in machine learning for tasks involving sequences, images, sets, and graphs, but the work is incremental as it builds on existing group theory tools.
The paper tackled the problem of proving the universality of equivariant multilayer perceptrons (MLPs) for learning on structured data, showing that a single hidden layer with a regular group action is sufficient for universal equivariance, with unconditional results for Abelian groups and bounds for high-order hidden layers.
Group invariant and equivariant Multilayer Perceptrons (MLP), also known as Equivariant Networks, have achieved remarkable success in learning on a variety of data structures, such as sequences, images, sets, and graphs. Using tools from group theory, this paper proves the universality of a broad class of equivariant MLPs with a single hidden layer. In particular, it is shown that having a hidden layer on which the group acts regularly is sufficient for universal equivariance (invariance). A corollary is unconditional universality of equivariant MLPs for Abelian groups, such as CNNs with a single hidden layer. A second corollary is the universality of equivariant MLPs with a high-order hidden layer, where we give both group-agnostic bounds and means for calculating group-specific bounds on the order of hidden layer that guarantees universal equivariance (invariance).