Approximately Equivariant Neural Processes
This work addresses the challenge of applying equivariant models to real-world data with imperfect symmetries, offering a flexible solution that could benefit domains like climate modeling, but it is incremental as it builds on existing equivariant architectures.
The authors tackled the problem of real-world data often being only approximately symmetric, which breaks exact equivariance in deep learning models, by developing a general method to make any equivariant architecture flexible enough to handle approximate symmetries. They applied this to neural processes and demonstrated improved performance over both non-equivariant and strictly equivariant models in synthetic and real-world regression tasks.
Equivariant deep learning architectures exploit symmetries in learning problems to improve the sample efficiency of neural-network-based models and their ability to generalise. However, when modelling real-world data, learning problems are often not exactly equivariant, but only approximately. For example, when estimating the global temperature field from weather station observations, local topographical features like mountains break translation equivariance. In these scenarios, it is desirable to construct architectures that can flexibly depart from exact equivariance in a data-driven way. Current approaches to achieving this cannot usually be applied out-of-the-box to any architecture and symmetry group. In this paper, we develop a general approach to achieving this using existing equivariant architectures. Our approach is agnostic to both the choice of symmetry group and model architecture, making it widely applicable. We consider the use of approximately equivariant architectures in neural processes (NPs), a popular family of meta-learning models. We demonstrate the effectiveness of our approach on a number of synthetic and real-world regression experiments, showing that approximately equivariant NP models can outperform both their non-equivariant and strictly equivariant counterparts.