Convolutional Neural Network Pruning Using Filter Attenuation
This work addresses the issue of maintaining network performance during pruning for researchers and practitioners in deep learning, though it is incremental as it builds on existing filter pruning techniques.
The paper tackles the problem of performance degradation and irreversibility in filter pruning for CNNs by proposing a filter attenuation method that gradually weakens filters instead of removing them abruptly, achieving better results compared to conventional pruning methods on the VGG model for Cifar10 classification.
Filters are the essential elements in convolutional neural networks (CNNs). Filters are corresponded to the feature maps and form the main part of the computational and memory requirement for the CNN processing. In filter pruning methods, a filter with all of its components, including channels and connections, are removed. The removal of a filter can cause a drastic change in the network's performance. Also, the removed filters cannot come back to the network structure. We want to address these problems in this paper. We propose a CNN pruning method based on filter attenuation in which weak filters are not directly removed. Instead, weak filters are attenuated and gradually removed. In the proposed attenuation approach, weak filters are not abruptly removed, and there is a chance for these filters to return to the network. The filter attenuation method is assessed using the VGG model for the Cifar10 image classification task. Simulation results show that the filter attenuation works with different pruning criteria, and better results are obtained in comparison with the conventional pruning methods.