Model Complexity-Accuracy Trade-off for a Convolutional Neural Network
This work addresses the bottleneck of deploying large CNNs on resource-constrained devices, but it is incremental as it builds on existing methods for model compression and trade-off analysis.
The paper tackles the problem of high memory footprint in Convolutional Neural Networks (CNNs) for deployment on devices with limited storage, such as mobile phones and IoT, by studying the trade-off between model complexity and accuracy on the MNIST dataset, resulting in a 236 times reduction in model complexity and a 19.5 times reduction in memory footprint while maintaining a worst-case accuracy threshold.
Convolutional Neural Networks(CNN) has had a great success in the recent past, because of the advent of faster GPUs and memory access. CNNs are really powerful as they learn the features from data in layers such that they exhibit the structure of the V-1 features of the human brain. A huge bottleneck, in this case, is that CNNs are very large and have a very high memory footprint, and hence they cannot be employed on devices with limited storage such as mobile phone, IoT etc. In this work, we study the model complexity versus accuracy trade-off on MNSIT dataset, and give a concrete framework for handling such a problem, given the worst case accuracy that a system can tolerate. In our work, we reduce the model complexity by 236 times, and memory footprint by 19.5 times compared to the base model while achieving worst case accuracy threshold.