Filter Pruning based on Information Capacity and Independence
This addresses the need for efficient and interpretable model compression in deep learning, though it is incremental as it builds on existing filter pruning techniques.
The paper tackles the problem of biased filter selection and high computational cost in filter pruning for CNNs by introducing a method that evaluates filters based on information capacity and independence, achieving a 77.4% reduction in FLOPs and 69.3% reduction in parameters for ResNet-50 with only a 2.64% accuracy drop.
Filter pruning has gained widespread adoption for the purpose of compressing and speeding up convolutional neural networks (CNNs). However, existing approaches are still far from practical applications due to biased filter selection and heavy computation cost. This paper introduces a new filter pruning method that selects filters in an interpretable, multi-perspective, and lightweight manner. Specifically, we evaluate the contributions of filters from both individual and overall perspectives. For the amount of information contained in each filter, a new metric called information capacity is proposed. Inspired by the information theory, we utilize the interpretable entropy to measure the information capacity, and develop a feature-guided approximation process. For correlations among filters, another metric called information independence is designed. Since the aforementioned metrics are evaluated in a simple but effective way, we can identify and prune the least important filters with less computation cost. We conduct comprehensive experiments on benchmark datasets employing various widely-used CNN architectures to evaluate the performance of our method. For instance, on ILSVRC-2012, our method outperforms state-of-the-art methods by reducing FLOPs by 77.4% and parameters by 69.3% for ResNet-50 with only a minor decrease in accuracy of 2.64%.