MorphoActivation: Generalizing ReLU activation function by mathematical morphology
This work addresses the need for more flexible activation functions in deep learning, but it appears incremental as it builds on existing morphological concepts without demonstrating broad breakthroughs.
The paper tackles the problem of generalizing ReLU activation functions in deep convolutional neural networks by using mathematical morphology, proposing a family of activation functions that combine max-pooling and nonlinear operators, and validates this approach with experiments on classical supervised learning benchmarks.
This paper analyses both nonlinear activation functions and spatial max-pooling for Deep Convolutional Neural Networks (DCNNs) by means of the algebraic basis of mathematical morphology. Additionally, a general family of activation functions is proposed by considering both max-pooling and nonlinear operators in the context of morphological representations. Experimental section validates the goodness of our approach on classical benchmarks for supervised learning by DCNN.