Automatic Pruning for Quantized Neural Networks
This work addresses the need for efficient model compression for deployment in resource-constrained environments, but it is incremental as it builds on existing quantization and pruning techniques.
The paper tackles the problem of pruning quantized neural networks, which is challenging due to discrete parameter distributions, by proposing an automatic pruning strategy that selects redundant low-precision filters and uses Bayesian optimization to determine layer-wise pruning ratios. It achieves a 26.12% reduction in model size for ResNet-18 on ImageNet with binarized quantization, resulting in a top-1 accuracy of 47.32% in a 2.47 MB model.
Neural network quantization and pruning are two techniques commonly used to reduce the computational complexity and memory footprint of these models for deployment. However, most existing pruning strategies operate on full-precision and cannot be directly applied to discrete parameter distributions after quantization. In contrast, we study a combination of these two techniques to achieve further network compression. In particular, we propose an effective pruning strategy for selecting redundant low-precision filters. Furthermore, we leverage Bayesian optimization to efficiently determine the pruning ratio for each layer. We conduct extensive experiments on CIFAR-10 and ImageNet with various architectures and precisions. In particular, for ResNet-18 on ImageNet, we prune 26.12% of the model size with Binarized Neural Network quantization, achieving a top-1 classification accuracy of 47.32% in a model of 2.47 MB and 59.30% with a 2-bit DoReFa-Net in 4.36 MB.