Variational Autoencoding Neural Operators
This work addresses the challenge of modeling functional data in applications like computer vision and climate modeling, but it is incremental as it adapts existing variational autoencoder concepts to operator learning.
The authors tackled the problem of unsupervised learning with functional data by introducing Variational Autoencoding Neural Operators (VANO), a method to make operator learning architectures act as variational autoencoders, and demonstrated its performance on benchmarks including Gaussian random fields, Cahn-Hilliard systems, and satellite data for Earth surface deformation.
Unsupervised learning with functional data is an emerging paradigm of machine learning research with applications to computer vision, climate modeling and physical systems. A natural way of modeling functional data is by learning operators between infinite dimensional spaces, leading to discretization invariant representations that scale independently of the sample grid resolution. Here we present Variational Autoencoding Neural Operators (VANO), a general strategy for making a large class of operator learning architectures act as variational autoencoders. For this purpose, we provide a novel rigorous mathematical formulation of the variational objective in function spaces for training. VANO first maps an input function to a distribution over a latent space using a parametric encoder and then decodes a sample from the latent distribution to reconstruct the input, as in classic variational autoencoders. We test VANO with different model set-ups and architecture choices for a variety of benchmarks. We start from a simple Gaussian random field where we can analytically track what the model learns and progressively transition to more challenging benchmarks including modeling phase separation in Cahn-Hilliard systems and real world satellite data for measuring Earth surface deformation.