Learning Efficient Convolutional Networks through Network Slimming
This addresses the deployment challenge of CNNs in real-world applications by making models more efficient, though it is incremental as it builds on existing pruning techniques.
The paper tackles the high computational cost of deep convolutional neural networks by proposing network slimming, a method that enforces channel-level sparsity to reduce model size, memory footprint, and operations without accuracy loss, achieving a 20x reduction in model size and 5x reduction in operations for VGGNet.
The deployment of deep convolutional neural networks (CNNs) in many real world applications is largely hindered by their high computational cost. In this paper, we propose a novel learning scheme for CNNs to simultaneously 1) reduce the model size; 2) decrease the run-time memory footprint; and 3) lower the number of computing operations, without compromising accuracy. This is achieved by enforcing channel-level sparsity in the network in a simple but effective way. Different from many existing approaches, the proposed method directly applies to modern CNN architectures, introduces minimum overhead to the training process, and requires no special software/hardware accelerators for the resulting models. We call our approach network slimming, which takes wide and large networks as input models, but during training insignificant channels are automatically identified and pruned afterwards, yielding thin and compact models with comparable accuracy. We empirically demonstrate the effectiveness of our approach with several state-of-the-art CNN models, including VGGNet, ResNet and DenseNet, on various image classification datasets. For VGGNet, a multi-pass version of network slimming gives a 20x reduction in model size and a 5x reduction in computing operations.