A scheme for automatic differentiation of complex loss functions
This work addresses a technical bottleneck for researchers and practitioners implementing complex-valued neural networks, though it is incremental as it builds on established real-function differentiation.
The paper tackles the lack of widespread automatic differentiation support for complex functions in machine learning by proposing an efficient and seamless scheme that generalizes existing real-function methods, simplifying the implementation of neural networks using complex numbers.
For a real function, automatic differentiation is such a standard algorithm used to efficiently compute its gradient, that it is integrated in various neural network frameworks. However, despite the recent advances in using complex functions in machine learning and the well-established usefulness of automatic differentiation, the support of automatic differentiation for complex functions is not as well-established and widespread as for real functions. In this work we propose an efficient and seamless scheme to implement automatic differentiation for complex functions, which is a compatible generalization of the current scheme for real functions. This scheme can significantly simplify the implementation of neural networks which use complex numbers.