GLGENN: A Novel Parameter-Light Equivariant Neural Networks Architecture Based on Clifford Geometric Algebras
This work addresses the need for more efficient and generalizable equivariant neural networks in machine learning, particularly for tasks involving geometric transformations, though it appears incremental as it builds on existing equivariant methods with a novel parametrization.
The authors tackled the problem of designing efficient equivariant neural networks by introducing GLGENN, a parameter-light architecture based on Clifford geometric algebras that is equivariant to pseudo-orthogonal transformations. The result shows that GLGENN outperforms or matches competitors on benchmarking tasks while using significantly fewer parameters, reducing overfitting tendencies.
We propose, implement, and compare with competitors a new architecture of equivariant neural networks based on geometric (Clifford) algebras: Generalized Lipschitz Group Equivariant Neural Networks (GLGENN). These networks are equivariant to all pseudo-orthogonal transformations, including rotations and reflections, of a vector space with any non-degenerate or degenerate symmetric bilinear form. We propose a weight-sharing parametrization technique that takes into account the fundamental structures and operations of geometric algebras. Due to this technique, GLGENN architecture is parameter-light and has less tendency to overfitting than baseline equivariant models. GLGENN outperforms or matches competitors on several benchmarking equivariant tasks, including estimation of an equivariant function and a convex hull experiment, while using significantly fewer optimizable parameters.