First Power Linear Unit with Sign
This work addresses the need for improved activation functions in deep learning, but it appears incremental as it builds upon existing methods with a new formulation.
The paper tackles the problem of designing a novel activation function for neural networks, proposing FPLUS, which uses a power function with polar signs, and shows it achieves superior competitiveness and stability across CNN architectures on benchmark datasets.
This paper proposes a novel and insightful activation method termed FPLUS, which exploits mathematical power function with polar signs in form. It is enlightened by common inverse operation while endowed with an intuitive meaning of bionics. The formulation is derived theoretically under conditions of some prior knowledge and anticipative properties, and then its feasibility is verified through a series of experiments using typical benchmark datasets, whose results indicate our approach owns superior competitiveness among numerous activation functions, as well as compatible stability across many CNN architectures. Furthermore, we extend the function presented to a more generalized type called PFPLUS with two parameters that can be fixed or learnable, so as to augment its expressive capacity, and outcomes of identical tests validate this improvement.