Rates and architectures for learning geometrically non-trivial operators
This work addresses a key challenge in scientific machine learning for problems involving complex geometry and singularity propagation, though it is incremental by extending existing theory to a broader class of operators.
The paper tackles the problem of learning geometric integral operators, such as generalized Radon transforms, which propagate singularities in scientific applications like waves and fluid dynamics, and proves that these operators can be learned with superalgebraic error decay, avoiding the curse of dimensionality, while proposing an architecture based on cross-attention and levelset methods that achieves this from very few training examples.
Deep learning methods have proven capable of recovering operators between high-dimensional spaces, such as solution maps of PDEs and similar objects in mathematical physics, from very few training samples. This phenomenon of data-efficiency has been proven for certain classes of elliptic operators with simple geometry, i.e., operators that do not change the domain of the function or propagate singularities. However, scientific machine learning is commonly used for problems that do involve the propagation of singularities in a priori unknown ways, such as waves, advection, and fluid dynamics. In light of this, we expand the learning theory to include double fibration transforms--geometric integral operators that include generalized Radon and geodesic ray transforms. We prove that this class of operators does not suffer from the curse of dimensionality: the error decays superalgebraically, that is, faster than any fixed power of the reciprocal of the number of training samples. Furthermore, we investigate architectures that explicitly encode the geometry of these transforms, demonstrating that an architecture reminiscent of cross-attention based on levelset methods yields a parameterization that is universal, stable, and learns double fibration transforms from very few training examples. Our results contribute to a rapidly-growing line of theoretical work on learning operators for scientific machine learning.