Probing Equivariance and Symmetry Breaking in Convolutional Networks
This work addresses the problem of optimizing structural priors in neural networks for researchers and practitioners, offering nuanced insights for model selection, though it is incremental in refining existing equivariance concepts.
The paper investigates the trade-offs of group equivariance in convolutional networks, finding that constrained equivariant models outperform less constrained ones when aligned with task geometry, and that symmetry-breaking techniques consistently improve performance across tasks like segmentation and generation.
In this work, we explore the trade-offs of explicit structural priors, particularly group equivariance. We address this through theoretical analysis and a comprehensive empirical study. To enable controlled and fair comparisons, we introduce \texttt{Rapidash}, a unified group convolutional architecture that allows for different variants of equivariant and non-equivariant models. Our results suggest that more constrained equivariant models outperform less constrained alternatives when aligned with the geometry of the task, and increasing representation capacity does not fully eliminate performance gaps. We see improved performance of models with equivariance and symmetry-breaking through tasks like segmentation, regression, and generation across diverse datasets. Explicit \textit{symmetry breaking} via geometric reference frames consistently improves performance, while \textit{breaking equivariance} through geometric input features can be helpful when aligned with task geometry. Our results provide task-specific performance trends that offer a more nuanced way for model selection.