WHC: Weighted Hybrid Criterion for Filter Pruning on Convolutional Neural Networks
This addresses the need for efficient compression and acceleration of CNNs, though it is incremental as it builds on existing norm-based and relationship-based criteria.
The paper tackles the problem of performance degradation in filter pruning for convolutional neural networks by proposing a new data-independent criterion, Weighted Hybrid Criterion (WHC), which robustly identifies redundant filters. It achieves a reduction of over 42% in floating point operations for ResNet-50 on ImageNet without loss in top-5 accuracy.
Filter pruning has attracted increasing attention in recent years for its capacity in compressing and accelerating convolutional neural networks. Various data-independent criteria, including norm-based and relationship-based ones, were proposed to prune the most unimportant filters. However, these state-of-the-art criteria fail to fully consider the dissimilarity of filters, and thus might lead to performance degradation. In this paper, we first analyze the limitation of relationship-based criteria with examples, and then introduce a new data-independent criterion, Weighted Hybrid Criterion (WHC), to tackle the problems of both norm-based and relationship-based criteria. By taking the magnitude of each filter and the linear dependence between filters into consideration, WHC can robustly recognize the most redundant filters, which can be safely pruned without introducing severe performance degradation to networks. Extensive pruning experiments in a simple one-shot manner demonstrate the effectiveness of the proposed WHC. In particular, WHC can prune ResNet-50 on ImageNet with more than 42% of floating point operations reduced without any performance loss in top-5 accuracy.