Universal Approximation and the Topological Neural Network
This work provides a theoretical foundation for neural networks to process complex data types like measures and topological spaces, which is incremental but extends deep learning to new mathematical domains.
The paper introduces topological neural networks (TNNs) that operate on Tychonoff topological spaces and distributional neural networks (DNNs) that handle Borel measures, enabling applications like analyzing stochastic processes or belief states. It proves a strong universal approximation theorem showing these networks can arbitrarily approximate uniformly continuous functions on such spaces.
A topological neural network (TNN), which takes data from a Tychonoff topological space instead of the usual finite dimensional space, is introduced. As a consequence, a distributional neural network (DNN) that takes Borel measures as data is also introduced. Combined these new neural networks facilitate things like recognizing long range dependence, heavy tails and other properties in stochastic process paths or like acting on belief states produced by particle filtering or hidden Markov model algorithms. The veracity of the TNN and DNN are then established herein by a strong universal approximation theorem for Tychonoff spaces and its corollary for spaces of measures. These theorems show that neural networks can arbitrarily approximate uniformly continuous functions (with respect to the sup metric) associated with a unique uniformity. We also provide some discussion showing that neural networks on positive-finite measures are a generalization of the recent deep learning notion of deep sets.