Stochastic Neural Network Symmetrisation in Markov Categories
This provides a mathematically precise method for symmetrising neural networks, which is useful for researchers working with equivariant models, though it appears incremental in extending existing techniques to stochastic cases.
The paper tackles the problem of symmetrising neural networks along group homomorphisms, providing a flexible compositional framework using Markov categories that works for both deterministic and stochastic models while recovering existing techniques.
We consider the problem of symmetrising a neural network along a group homomorphism: given a homomorphism $\varphi : H \to G$, we would like a procedure that converts $H$-equivariant neural networks to $G$-equivariant ones. We formulate this in terms of Markov categories, which allows us to consider neural networks whose outputs may be stochastic, but with measure-theoretic details abstracted away. We obtain a flexible and compositional framework for symmetrisation that relies on minimal assumptions about the structure of the group and the underlying neural network architecture. Our approach recovers existing canonicalisation and averaging techniques for symmetrising deterministic models, and extends to provide a novel methodology for symmetrising stochastic models also. Beyond this, our findings also demonstrate the utility of Markov categories for addressing complex problems in machine learning in a conceptually clear yet mathematically precise way.