On the Implicit Bias of Linear Equivariant Steerable Networks
This work addresses the problem of understanding and improving generalization in invariant machine learning models, providing theoretical insights for researchers in equivariant neural networks, though it is incremental as it builds on existing implicit bias and steerable network frameworks.
The paper investigates the implicit bias of gradient flow in linear equivariant steerable networks for group-invariant binary classification, showing that the predictor converges to a maximum margin classifier defined by the input group action and that steerable networks achieve improved margin and generalization bounds compared to non-invariant networks.
We study the implicit bias of gradient flow on linear equivariant steerable networks in group-invariant binary classification. Our findings reveal that the parameterized predictor converges in direction to the unique group-invariant classifier with a maximum margin defined by the input group action. Under a unitary assumption on the input representation, we establish the equivalence between steerable networks and data augmentation. Furthermore, we demonstrate the improved margin and generalization bound of steerable networks over their non-invariant counterparts.