Neural Networks with Complex-Valued Weights Have No Spurious Local Minima
This addresses optimization challenges in neural networks for researchers, offering a theoretical advantage for complex-valued models, though it is incremental as it focuses on a specific activation and shallow architecture.
The paper proves that shallow complex neural networks with quadratic activations have no spurious local minima, unlike real networks which have infinitely many under the same conditions, and provides examples showing complex weights turn poor minima into saddle points.
We study the benefits of complex-valued weights for neural networks. We prove that shallow complex neural networks with quadratic activations have no spurious local minima. In contrast, shallow real neural networks with quadratic activations have infinitely many spurious local minima under the same conditions. In addition, we provide specific examples to demonstrate that complex-valued weights turn poor local minima into saddle points.