Learning Equivariant Functions via Quadratic Forms
This work addresses the challenge of incorporating symmetry into machine learning models for researchers in fields like physics and computer vision, though it appears incremental as it builds on known properties of orthogonal groups.
The authors tackled the problem of learning group-equivariant functions by introducing a method that learns the associated quadratic form from data, assuming the underlying symmetry group is orthogonal. Their approach led to models that are simplified and efficient, consistently excelling in discovering underlying symmetries and learning equivariant functions across tasks like polynomial regression, top quark tagging, and moment of inertia matrix prediction.
In this study, we introduce a method for learning group (known or unknown) equivariant functions by learning the associated quadratic form $x^T A x$ corresponding to the group from the data. Certain groups, known as orthogonal groups, preserve a specific quadratic form, and we leverage this property to uncover the underlying symmetry group under the assumption that it is orthogonal. By utilizing the corresponding unique symmetric matrix and its inherent diagonal form, we incorporate suitable inductive biases into the neural network architecture, leading to models that are both simplified and efficient. Our approach results in an invariant model that preserves norms, while the equivariant model is represented as a product of a norm-invariant model and a scale-invariant model, where the ``product'' refers to the group action. Moreover, we extend our framework to a more general setting where the function acts on tuples of input vectors via a diagonal (or product) group action. In this extension, the equivariant function is decomposed into an angular component extracted solely from the normalized first vector and a scale-invariant component that depends on the full Gram matrix of the tuple. This decomposition captures the inter-dependencies between multiple inputs while preserving the underlying group symmetry. We assess the effectiveness of our framework across multiple tasks, including polynomial regression, top quark tagging, and moment of inertia matrix prediction. Comparative analysis with baseline methods demonstrates that our model consistently excels in both discovering the underlying symmetry and efficiently learning the corresponding equivariant function.