Structured Pruning for Efficient ConvNets via Incremental Regularization
This work addresses the need for more efficient CNN deployment in resource-constrained environments, but it is incremental as it builds on existing regularization-based pruning approaches.
The paper tackles the problem of efficiently compressing and accelerating convolutional neural networks (CNNs) through structured pruning, proposing a method called IncReg that incrementally assigns regularization factors based on weight importance, achieving competitive results with state-of-the-art methods on popular CNNs.
Parameter pruning is a promising approach for CNN compression and acceleration by eliminating redundant model parameters with tolerable performance loss. Despite its effectiveness, existing regularization-based parameter pruning methods usually drive weights towards zero with large and constant regularization factors, which neglects the fact that the expressiveness of CNNs is fragile and needs a more gentle way of regularization for the networks to adapt during pruning. To solve this problem, we propose a new regularization-based pruning method (named IncReg) to incrementally assign different regularization factors to different weight groups based on their relative importance, whose effectiveness is proved on popular CNNs compared with state-of-the-art methods.