Universality of shallow and deep neural networks on non-Euclidean spaces
For researchers in approximation theory and geometric deep learning, it provides a theoretical foundation for using neural networks on non-Euclidean data, though the results are largely theoretical and incremental.
This paper extends universal approximation theorems for neural networks from Euclidean spaces to general topological spaces, proving that shallow and deep narrow networks can approximate continuous functions on non-Euclidean domains under certain conditions, with explicit width bounds for products of compact metric spaces.
We study shallow and deep neural networks whose inputs range over a general topological space. The model is built from a prescribed family of continuous feature maps and reduces to multilayer feedforward networks in the Euclidean case. We focus on the universal approximation property and establish general conditions under which such networks are dense in spaces of continuous vector-valued functions on arbitrary topological spaces and, in particular, locally convex spaces. Universality results obtained in the arbitrary-width case extend classical approximation theorems to non-Euclidean spaces. We also consider the deep narrow setting, in which the width of each hidden layer is uniformly bounded while the depth is allowed to grow. We identify conditions under which such networks retain the universal approximation property. As a concrete example, we employ Ostrand's extension of the Kolmogorov superposition theorem to derive an explicit universality result for products of compact metric spaces, with width bounds expressed in terms of topological dimension.