Complex-valued Neural Networks -- Theory and Analysis
It provides a comprehensive overview for researchers working on wave-typed information and frequency-domain processing, but is incremental as it synthesizes existing knowledge without introducing new methods or results.
This work addresses the theory and analysis of complex-valued neural networks (CVNNs), covering structures, classification, activation functions, learning algorithms, and implementation modules to understand their dynamics and recent developments.
Complex-valued neural networks (CVNNs) have recently been successful in various pioneering areas which involve wave-typed information and frequency-domain processing. This work addresses different structures and classification of CVNNs. The theory behind complex activation functions, implications related to complex differentiability and special activations for CVNN output layers are presented. The work also discusses CVNN learning and optimization using gradient and non-gradient based algorithms. Complex Backpropagation utilizing complex chain rule is also explained in terms of Wirtinger calculus. Moreover, special modules for building CVNN models, such as complex batch normalization and complex random initialization are also discussed. The work also highlights libraries and software blocks proposed for CVNN implementations and discusses future directions. The objective of this work is to understand the dynamics and most recent developments of CVNNs.