Quantitative approximation results for complex-valued neural networks
This work addresses the lack of theoretical understanding for CVNNs, which are important for applications like MRI fingerprinting, but it is incremental as it builds on existing real-valued network theory.
The paper tackles the problem of quantifying the approximation capabilities of complex-valued neural networks (CVNNs) using the modReLU activation function, showing that the derived error bounds for approximating smooth functions on compact subsets are optimal up to logarithmic factors.
Until recently, applications of neural networks in machine learning have almost exclusively relied on real-valued networks. It was recently observed, however, that complex-valued neural networks (CVNNs) exhibit superior performance in applications in which the input is naturally complex-valued, such as MRI fingerprinting. While the mathematical theory of real-valued networks has, by now, reached some level of maturity, this is far from true for complex-valued networks. In this paper, we analyze the expressivity of complex-valued networks by providing explicit quantitative error bounds for approximating $C^n$ functions on compact subsets of $\mathbb{C}^d$ by complex-valued neural networks that employ the modReLU activation function, given by $σ(z) = \mathrm{ReLU}(|z| - 1) \, \mathrm{sgn} (z)$, which is one of the most popular complex activation functions used in practice. We show that the derived approximation rates are optimal (up to log factors) in the class of modReLU networks with weights of moderate growth.