Neural Flow Operators can Approximate any Operator: Abstract Frameworks and Universal Approcimations
This provides a theoretical foundation for flow-based models, unifying residual and plain architectures, which is significant for the deep learning community.
The paper introduces an abstract neural flow framework for neural networks and operators, proving universal approximation properties for flow-based models in both finite and infinite-dimensional spaces, including the first such result for infinite-dimensional flows.
We introduce an abstract neural flow framework for neural networks and neural operators. The framework contains two continuous-depth models, namely neural flows with composition and separation structures, and covers both finite-dimensional function approximation and infinite-dimensional operator approximation. We prove well-posedness and universal approximation properties for the corresponding neural flows, including, to the best of our knowledge, the first universal approximation result for flow-based models between infinite-dimensional spaces. We also obtain universal approximation results for convolutional neural flow models. Through suitable time discretizations, the composition structure recovers ResNet-type architectures, while the separation structure, via a splitting-based discretization, yields plain architectures. This gives a unified flow-based route to both residual and plain architectures for neural networks and neural operators with fully connected or convolutional linear layers.