Extending the Universal Approximation Theorem for a Broad Class of Hypercomplex-Valued Neural Networks
This foundational result supports the theoretical use of hypercomplex-valued neural networks in applications like regression and classification, but it is incremental as it generalizes existing theorems to a broader class of algebras.
The paper extends the universal approximation theorem to a broad class of hypercomplex-valued neural networks, proving that networks based on non-degenerate hypercomplex algebras can approximate continuous functions with any desired precision on compact sets.
The universal approximation theorem asserts that a single hidden layer neural network approximates continuous functions with any desired precision on compact sets. As an existential result, the universal approximation theorem supports the use of neural networks for various applications, including regression and classification tasks. The universal approximation theorem is not limited to real-valued neural networks but also holds for complex, quaternion, tessarines, and Clifford-valued neural networks. This paper extends the universal approximation theorem for a broad class of hypercomplex-valued neural networks. Precisely, we first introduce the concept of non-degenerate hypercomplex algebra. Complex numbers, quaternions, and tessarines are examples of non-degenerate hypercomplex algebras. Then, we state the universal approximation theorem for hypercomplex-valued neural networks defined on a non-degenerate algebra.