Universal Approximation of Continuous Functionals on Compact Subsets via Linear Measurements and Scalar Nonlinearities
This provides a theoretical justification for a common design pattern in operator learning and imaging, addressing a foundational mathematical problem in functional analysis.
The paper tackles the problem of approximating continuous functionals on compact subsets of Hilbert spaces, proving that any such functional can be uniformly approximated using finitely many linear measurements combined with scalar nonlinearities, and extends this to maps with values in Banach spaces for finite-rank approximations.
We study universal approximation of continuous functionals on compact subsets of products of Hilbert spaces. We prove that any such functional can be uniformly approximated by models that first take finitely many continuous linear measurements of the inputs and then combine these measurements through continuous scalar nonlinearities. We also extend the approximation principle to maps with values in a Banach space, yielding finite-rank approximations. These results provide a compact-set justification for the common ``measure, apply scalar nonlinearities, then combine'' design pattern used in operator learning and imaging.