Learning Symmetrization for Equivariance with Orbit Distance Minimization
This work addresses the need for more general equivariant neural networks in machine learning, though it builds incrementally on existing symmetrization proposals.
The authors tackled the problem of making arbitrary neural networks equivariant to given groups by introducing a symmetrization framework that replaces prior feature conversion methods with an orbit distance minimization loss, enabling broader applicability to groups like O(1,3). They demonstrated competitiveness on SO(2) image classification and increased generality on O(1,3) tasks.
We present a general framework for symmetrizing an arbitrary neural-network architecture and making it equivariant with respect to a given group. We build upon the proposals of Kim et al. (2023); Kaba et al. (2023) for symmetrization, and improve them by replacing their conversion of neural features into group representations, with an optimization whose loss intuitively measures the distance between group orbits. This change makes our approach applicable to a broader range of matrix groups, such as the Lorentz group O(1, 3), than these two proposals. We experimentally show our method's competitiveness on the SO(2) image classification task, and also its increased generality on the task with O(1, 3). Our implementation will be made accessible at https://github.com/tiendatnguyen-vision/Orbit-symmetrize.