Pruning of Convolutional Neural Networks Using Ising Energy Model
This addresses the need for more efficient neural networks in resource-constrained applications, though it is incremental as it builds on existing pruning methods.
The paper tackled the problem of compressing deep neural networks by proposing an Ising energy model for pruning convolutional kernels and hidden units, achieving over 50% pruning rate with less than 10% and 5% drops in Top-1 and Top-5 accuracy on CIFAR datasets.
Pruning is one of the major methods to compress deep neural networks. In this paper, we propose an Ising energy model within an optimization framework for pruning convolutional kernels and hidden units. This model is designed to reduce redundancy between weight kernels and detect inactive kernels/hidden units. Our experiments using ResNets, AlexNet, and SqueezeNet on CIFAR-10 and CIFAR-100 datasets show that the proposed method on average can achieve a pruning rate of more than $50\%$ of the trainable parameters with approximately $<10\%$ and $<5\%$ drop of Top-1 and Top-5 classification accuracy, respectively.