A Computationally Efficient Neural Network Invariant to the Action of Symmetry Subgroups
This work addresses the need for efficient invariant networks in machine learning, offering a novel method for handling symmetry subgroups, though it is incremental in the context of existing invariant network research.
The paper tackles the problem of designing computationally efficient neural networks invariant to permutation subgroups by introducing a G-invariant transformation module that produces invariant latent representations, achieving strong generalization and efficiency in numerical experiments.
We introduce a method to design a computationally efficient $G$-invariant neural network that approximates functions invariant to the action of a given permutation subgroup $G \leq S_n$ of the symmetric group on input data. The key element of the proposed network architecture is a new $G$-invariant transformation module, which produces a $G$-invariant latent representation of the input data. This latent representation is then processed with a multi-layer perceptron in the network. We prove the universality of the proposed architecture, discuss its properties and highlight its computational and memory efficiency. Theoretical considerations are supported by numerical experiments involving different network configurations, which demonstrate the effectiveness and strong generalization properties of the proposed method in comparison to other $G$-invariant neural networks.