Compact and Efficient Neural Networks for Image Recognition Based on Learned 2D Separable Transform
This addresses the need for compact and efficient neural networks for image recognition, particularly in resource-constrained environments, though it is incremental as it builds on existing separable transform ideas.
The paper tackles the problem of reducing neural network parameters for image recognition by introducing a learned two-dimensional separable transform (LST) layer, achieving 98.02% accuracy on MNIST with only 9.5k parameters and demonstrating efficiency on FPGA.
The paper presents a learned two-dimensional separable transform (LST) that can be considered as a new type of computational layer for constructing neural network (NN) architecture for image recognition tasks. The LST based on the idea of sharing the weights of one fullyconnected (FC) layer to process all rows of an image. After that, a second shared FC layer is used to process all columns of image representation obtained from the first layer. The use of LST layers in a NN architecture significantly reduces the number of model parameters compared to models that use stacked FC layers. We show that a NN-classifier based on a single LST layer followed by an FC layer achieves 98.02\% accuracy on the MNIST dataset, while having only 9.5k parameters. We also implemented a LST-based classifier for handwritten digit recognition on the FPGA platform to demonstrate the efficiency of the suggested approach for designing a compact and high-performance implementation of NN models. Git repository with supplementary materials: https://github.com/Mak-Sim/LST-2d