Structured Pruning for Efficient ConvNets via Incremental Regularization
This work addresses the need for more efficient CNN models for deployment in resource-constrained environments, presenting an incremental improvement over existing pruning techniques.
The paper tackles the problem of efficiently compressing and accelerating convolutional neural networks (CNNs) through structured pruning, proposing a new regularization-based method called IncReg that incrementally assigns different regularization factors to weights based on their importance, achieving comparable or better results than state-of-the-art methods on CIFAR-10 and ImageNet datasets.
Parameter pruning is a promising approach for CNN compression and acceleration by eliminating redundant model parameters with tolerable performance degrade. Despite its effectiveness, existing regularization-based parameter pruning methods usually drive weights towards zero with large and constant regularization factors, which neglects the fragility of the expressiveness of CNNs, and thus calls for a more gentle regularization scheme so that the networks can adapt during pruning. To achieve this, we propose a new and novel regularization-based pruning method, named IncReg, to incrementally assign different regularization factors to different weights based on their relative importance. Empirical analysis on CIFAR-10 dataset verifies the merits of IncReg. Further extensive experiments with popular CNNs on CIFAR-10 and ImageNet datasets show that IncReg achieves comparable to even better results compared with state-of-the-arts. Our source codes and trained models are available here: https://github.com/mingsun-tse/caffe_increg.