A Generalization Method of Partitioned Activation Function for Complex Number
This work addresses the lack of well-established complex activation functions for complex number artificial neural networks, which is an incremental step in enabling better handling of complex number problems.
The authors tackled the problem of extending partitioned activation functions from real to complex numbers, proposing a method with four variations that can achieve holomorphic activation, conserve complex angle, or guarantee interaction between real and imaginary parts, as demonstrated with LReLU and SELU.
A method to convert real number partitioned activation function into complex number one is provided. The method has 4em variations; 1 has potential to get holomorphic activation, 2 has potential to conserve complex angle, and the last 1 guarantees interaction between real and imaginary parts. The method has been applied to LReLU and SELU as examples. The complex number activation function is an building block of complex number ANN, which has potential to properly deal with complex number problems. But the complex activation is not well established yet. Therefore, we propose a way to extend the partitioned real activation to complex number.