Deep Neural Networks Are Effective At Learning High-Dimensional Hilbert-Valued Functions From Limited Data
This work provides a theoretical foundation and practical insights for researchers and practitioners using DNNs to solve high-dimensional, Hilbert-valued function approximation problems, particularly in scientific computing and parametric PDEs. It offers an incremental understanding of DNN capabilities in this specific domain.
This paper investigates the effectiveness of Deep Neural Networks (DNNs) in approximating Hilbert-valued functions, which are common in science and engineering problems like parametric Partial Differential Equations (PDEs). The authors present a theoretical analysis of a non-standard DNN training procedure for holomorphic functions, demonstrating that DNNs can perform comparably to existing best-in-class schemes, establishing a benchmark lower bound. They also provide preliminary numerical results on parametric PDEs, exploring architectural modifications to achieve competitive performance.
Accurate approximation of scalar-valued functions from sample points is a key task in computational science. Recently, machine learning with Deep Neural Networks (DNNs) has emerged as a promising tool for scientific computing, with impressive results achieved on problems where the dimension of the data or problem domain is large. This work broadens this perspective, focusing on approximating functions that are Hilbert-valued, i.e. take values in a separable, but typically infinite-dimensional, Hilbert space. This arises in science and engineering problems, in particular those involving solution of parametric Partial Differential Equations (PDEs). Such problems are challenging: 1) pointwise samples are expensive to acquire, 2) the function domain is high dimensional, and 3) the range lies in a Hilbert space. Our contributions are twofold. First, we present a novel result on DNN training for holomorphic functions with so-called hidden anisotropy. This result introduces a DNN training procedure and full theoretical analysis with explicit guarantees on error and sample complexity. The error bound is explicit in three key errors occurring in the approximation procedure: the best approximation, measurement, and physical discretization errors. Our result shows that there exists a procedure (albeit non-standard) for learning Hilbert-valued functions via DNNs that performs as well as, but no better than current best-in-class schemes. It gives a benchmark lower bound for how well DNNs can perform on such problems. Second, we examine whether better performance can be achieved in practice through different types of architectures and training. We provide preliminary numerical results illustrating practical performance of DNNs on parametric PDEs. We consider different parameters, modifying the DNN architecture to achieve better and competitive results, comparing these to current best-in-class schemes.