Deep Nonparametric Estimation of Operators between Infinite Dimensional Spaces
This work addresses a foundational challenge in machine learning and scientific computing by providing theoretical guarantees for operator estimation, though it is incremental as it builds on existing nonparametric and neural network frameworks.
The paper tackles the problem of learning operators between infinite-dimensional spaces using deep neural networks, deriving non-asymptotic generalization error bounds that decay with sample size and depend on the intrinsic dimension of the data, with results applicable to real-world scenarios.
Learning operators between infinitely dimensional spaces is an important learning task arising in wide applications in machine learning, imaging science, mathematical modeling and simulations, etc. This paper studies the nonparametric estimation of Lipschitz operators using deep neural networks. Non-asymptotic upper bounds are derived for the generalization error of the empirical risk minimizer over a properly chosen network class. Under the assumption that the target operator exhibits a low dimensional structure, our error bounds decay as the training sample size increases, with an attractive fast rate depending on the intrinsic dimension in our estimation. Our assumptions cover most scenarios in real applications and our results give rise to fast rates by exploiting low dimensional structures of data in operator estimation. We also investigate the influence of network structures (e.g., network width, depth, and sparsity) on the generalization error of the neural network estimator and propose a general suggestion on the choice of network structures to maximize the learning efficiency quantitatively.