Categorical Equivariant Deep Learning: Category-Equivariant Neural Networks and Universal Approximation Theorems
This work expands equivariant deep learning beyond group actions to include contextual and compositional symmetries, providing a foundational framework for researchers in machine learning and related fields.
The authors tackled the problem of unifying various equivariant neural network frameworks by developing a theory of category-equivariant neural networks (CENNs), proving an equivariant universal approximation theorem that shows these networks can approximate continuous equivariant transformations.
We develop a theory of category-equivariant neural networks (CENNs) that unifies group/groupoid-equivariant networks, poset/lattice-equivariant networks, graph and sheaf neural networks. Equivariance is formulated as naturality in a topological category with Radon measures. Formulating linear and nonlinear layers in the categorical setup, we prove the equivariant universal approximation theorem in the general setting: the class of finite-depth CENNs is dense in the space of continuous equivariant transformations. We instantiate the framework for groups/groupoids, posets/lattices, graphs and cellular sheaves, deriving universal approximation theorems for them in a systematic manner. Categorical equivariant deep learning thus allows us to expand the horizons of equivariant deep learning beyond group actions, encompassing not only geometric symmetries but also contextual and compositional symmetries.