Evaluating CNN with Oscillatory Activation Function
This work addresses the problem of selecting activation functions for CNNs in image classification, but it is incremental as it tests an existing method on standard datasets without broad implications.
The paper evaluated the performance of AlexNet on MNIST and CIFAR10 datasets using an oscillatory activation function (GCU) compared to common functions like ReLU, PReLU, and Mish, finding that GCU improved accuracy by 2-3% on CIFAR10 but showed minimal gains on MNIST.
The reason behind CNNs capability to learn high-dimensional complex features from the images is the non-linearity introduced by the activation function. Several advanced activation functions have been discovered to improve the training process of neural networks, as choosing an activation function is a crucial step in the modeling. Recent research has proposed using an oscillating activation function to solve classification problems inspired by the human brain cortex. This paper explores the performance of one of the CNN architecture ALexNet on MNIST and CIFAR10 datasets using oscillatory activation function (GCU) and some other commonly used activation functions like ReLu, PReLu, and Mish.