CoShNet: A Hybrid Complex Valued Neural Network using Shearlets
This work addresses efficiency issues in deep learning for image classification, offering a domain-specific improvement with incremental novelty.
The authors tackled the problem of high computational cost and parameter count in convolutional neural networks by proposing CoShNet, a hybrid complex-valued neural network using shearlets, which achieved 92.2% accuracy on Fashion-MNIST with 49.9k parameters and 52 times fewer FLOPs compared to ResNet-18.
In a hybrid neural network, the expensive convolutional layers are replaced by a non-trainable fixed transform with a great reduction in parameters. In previous works, good results were obtained by replacing the convolutions with wavelets. However, wavelet based hybrid network inherited wavelet's lack of vanishing moments along curves and its axis-bias. We propose to use Shearlets with its robust support for important image features like edges, ridges and blobs. The resulting network is called Complex Shearlets Network (CoShNet). It was tested on Fashion-MNIST against ResNet-50 and Resnet-18, obtaining 92.2% versus 90.7% and 91.8% respectively. The proposed network has 49.9k parameters versus ResNet-18 with 11.18m and use 52 times fewer FLOPs. Finally, we trained in under 20 epochs versus 200 epochs required by ResNet and do not need any hyperparameter tuning nor regularization. Code: https://github.com/Ujjawal-K-Panchal/coshnet