Universal Approximation Theorem for Vector- and Hypercomplex-Valued Neural Networks
This work provides a theoretical foundation for using vector-valued neural networks in applications like regression and classification, but it is incremental as it builds on existing theorems for real- and hypercomplex-valued networks.
The paper extends the universal approximation theorem to a broad class of vector-valued neural networks, including hypercomplex-valued models, by introducing the concept of non-degenerate algebras and proving the theorem for networks defined on such structures.
The universal approximation theorem states that a neural network with one hidden layer can approximate continuous functions on compact sets with any desired precision. This theorem supports using neural networks for various applications, including regression and classification tasks. Furthermore, it is valid for real-valued neural networks and some hypercomplex-valued neural networks such as complex-, quaternion-, tessarine-, and Clifford-valued neural networks. However, hypercomplex-valued neural networks are a type of vector-valued neural network defined on an algebra with additional algebraic or geometric properties. This paper extends the universal approximation theorem for a wide range of vector-valued neural networks, including hypercomplex-valued models as particular instances. Precisely, we introduce the concept of non-degenerate algebra and state the universal approximation theorem for neural networks defined on such algebras.