Smaller Models, Better Generalization
This work addresses the need for efficient models in mobile technology, but it appears incremental as it builds on existing regularization and pruning techniques.
The paper tackles the problem of reducing neural network complexity for mobile vision tasks by analyzing methods like VC dimension bounds, pruning, and quantization, and proposes a novel loss function that achieves sparser models with comparable accuracy to dense ones.
Reducing network complexity has been a major research focus in recent years with the advent of mobile technology. Convolutional Neural Networks that perform various vision tasks without memory overhaul is the need of the hour. This paper focuses on qualitative and quantitative analysis of reducing the network complexity using an upper bound on the Vapnik-Chervonenkis dimension, pruning, and quantization. We observe a general trend in improvement of accuracies as we quantize the models. We propose a novel loss function that helps in achieving considerable sparsity at comparable accuracies to that of dense models. We compare various regularizations prevalent in the literature and show the superiority of our method in achieving sparser models that generalize well.