Universal approximations of permutation invariant/equivariant functions by deep neural networks
This work addresses the challenge of efficiently modeling symmetric functions in machine learning, which is incremental as it builds on representation theory to reduce parameter counts.
The paper tackles the problem of approximating permutation invariant/equivariant functions using deep neural networks by constructing universal approximators with layers that incorporate group actions, resulting in models with exponentially fewer free parameters than standard approaches.
In this paper, we develop a theory about the relationship between $G$-invariant/equivariant functions and deep neural networks for finite group $G$. Especially, for a given $G$-invariant/equivariant function, we construct its universal approximator by deep neural network whose layers equip $G$-actions and each affine transformations are $G$-equivariant/invariant. Due to representation theory, we can show that this approximator has exponentially fewer free parameters than usual models.