Decomposition of Equivariant Maps via Invariant Maps: Application to Universal Approximation under Symmetry
This work addresses the challenge of designing efficient neural networks with group symmetries, which is important for applications in fields like physics and computer vision, but it is incremental as it builds on existing theories of equivariant and invariant maps.
The paper tackles the problem of understanding and constructing equivariant neural networks by establishing a one-to-one relationship between equivariant and invariant maps, reducing arguments between them. As a result, it proposes a construction for universal equivariant architectures from invariant networks and provides an approximation rate for G-equivariant deep neural networks with ReLU activations for finite groups.
In this paper, we develop a theory about the relationship between invariant and equivariant maps with regard to a group $G$. We then leverage this theory in the context of deep neural networks with group symmetries in order to obtain novel insight into their mechanisms. More precisely, we establish a one-to-one relationship between equivariant maps and certain invariant maps. This allows us to reduce arguments for equivariant maps to those for invariant maps and vice versa. As an application, we propose a construction of universal equivariant architectures built from universal invariant networks. We, in turn, explain how the universal architectures arising from our construction differ from standard equivariant architectures known to be universal. Furthermore, we explore the complexity, in terms of the number of free parameters, of our models, and discuss the relation between invariant and equivariant networks' complexity. Finally, we also give an approximation rate for G-equivariant deep neural networks with ReLU activation functions for finite group G.