Filter Pruning using Hierarchical Group Sparse Regularization for Deep Convolutional Neural Networks
This addresses the need for more compact and efficient deep learning models, particularly for resource-constrained applications, though it is incremental as it builds on existing pruning and regularization techniques.
The paper tackles the problem of redundant parameters in convolutional neural networks by proposing a filter pruning method using hierarchical group sparse regularization, achieving over 50% parameter reduction in ResNet for CIFAR-10 with only a 0.3% accuracy drop and 34% reduction for TinyImageNet-200 with higher accuracy than baseline.
Since the convolutional neural networks are often trained with redundant parameters, it is possible to reduce redundant kernels or filters to obtain a compact network without dropping the classification accuracy. In this paper, we propose a filter pruning method using the hierarchical group sparse regularization. It is shown in our previous work that the hierarchical group sparse regularization is effective in obtaining sparse networks in which filters connected to unnecessary channels are automatically close to zero. After training the convolutional neural network with the hierarchical group sparse regularization, the unnecessary filters are selected based on the increase of the classification loss of the randomly selected training samples to obtain a compact network. It is shown that the proposed method can reduce more than 50% parameters of ResNet for CIFAR-10 with only 0.3% decrease in the accuracy of test samples. Also, 34% parameters of ResNet are reduced for TinyImageNet-200 with higher accuracy than the baseline network.