Image classification in frequency domain with 2SReLU: a second harmonics superposition activation function
This work addresses the challenge of frequency domain analysis in CNNs for image classification, offering an incremental improvement with a new activation function.
The authors tackled the problem of preserving high-frequency components in deep convolutional neural networks for image classification by proposing a novel activation function, 2SReLU, and achieved competitive results without using computationally costly convolution operations.
Deep Convolutional Neural Networks are able to identify complex patterns and perform tasks with super-human capabilities. However, besides the exceptional results, they are not completely understood and it is still impractical to hand-engineer similar solutions. In this work, an image classification Convolutional Neural Network and its building blocks are described from a frequency domain perspective. Some network layers have established counterparts in the frequency domain like the convolutional and pooling layers. We propose the 2SReLU layer, a novel non-linear activation function that preserves high frequency components in deep networks. It is demonstrated that in the frequency domain it is possible to achieve competitive results without using the computationally costly convolution operation. A source code implementation in PyTorch is provided at: https://gitlab.com/thomio/2srelu