Universal Functional Regression with Neural Operator Flows
This work addresses the limitation of Gaussian process priors in functional regression, offering a method for non-Gaussian function spaces, which is incremental as it extends normalizing flows to infinite dimensions.
The authors tackled the problem of regression on non-Gaussian function spaces by introducing Neural Operator Flows (OpFlow), an infinite-dimensional extension of normalizing flows that maps unknown function spaces into Gaussian processes for exact likelihood estimation, enabling robust uncertainty quantification and showing empirical performance on synthetic and real-world data like earthquake seismograms.
Regression on function spaces is typically limited to models with Gaussian process priors. We introduce the notion of universal functional regression, in which we aim to learn a prior distribution over non-Gaussian function spaces that remains mathematically tractable for functional regression. To do this, we develop Neural Operator Flows (OpFlow), an infinite-dimensional extension of normalizing flows. OpFlow is an invertible operator that maps the (potentially unknown) data function space into a Gaussian process, allowing for exact likelihood estimation of functional point evaluations. OpFlow enables robust and accurate uncertainty quantification via drawing posterior samples of the Gaussian process and subsequently mapping them into the data function space. We empirically study the performance of OpFlow on regression and generation tasks with data generated from Gaussian processes with known posterior forms and non-Gaussian processes, as well as real-world earthquake seismograms with an unknown closed-form distribution.