Network Pruning using Adaptive Exemplar Filters
This addresses the computational bottleneck in pruning neural networks for deployment, though it appears incremental as it builds on existing pruning concepts with a novel adaptation.
The paper tackles the problem of inefficient and suboptimal network pruning by introducing adaptive exemplar filters, resulting in EPruner which achieves significant FLOPs reduction (e.g., 76.34% on VGGNet-16) with minimal accuracy loss (e.g., 0.06% improvement on CIFAR-10).
Popular network pruning algorithms reduce redundant information by optimizing hand-crafted models, and may cause suboptimal performance and long time in selecting filters. We innovatively introduce adaptive exemplar filters to simplify the algorithm design, resulting in an automatic and efficient pruning approach called EPruner. Inspired by the face recognition community, we use a message passing algorithm Affinity Propagation on the weight matrices to obtain an adaptive number of exemplars, which then act as the preserved filters. EPruner breaks the dependency on the training data in determining the "important" filters and allows the CPU implementation in seconds, an order of magnitude faster than GPU based SOTAs. Moreover, we show that the weights of exemplars provide a better initialization for the fine-tuning. On VGGNet-16, EPruner achieves a 76.34%-FLOPs reduction by removing 88.80% parameters, with 0.06% accuracy improvement on CIFAR-10. In ResNet-152, EPruner achieves a 65.12%-FLOPs reduction by removing 64.18% parameters, with only 0.71% top-5 accuracy loss on ILSVRC-2012. Our code can be available at https://github.com/lmbxmu/EPruner.