Automatic Complementary Separation Pruning Toward Lightweight CNNs
This work addresses the need for lightweight CNNs in real-world deployment by automating pruning to eliminate manual tuning, though it is incremental as it builds on existing structured and activation-based pruning techniques.
The paper tackles the problem of reducing computational costs in convolutional neural networks by introducing Automatic Complementary Separation Pruning (ACSP), a fully automated pruning method that removes redundant components while maintaining competitive accuracy, as demonstrated on architectures like VGG-16 and ResNet-50 across datasets such as CIFAR-10 and ImageNet-1K.
In this paper, we present Automatic Complementary Separation Pruning (ACSP), a novel and fully automated pruning method for convolutional neural networks. ACSP integrates the strengths of both structured pruning and activation-based pruning, enabling the efficient removal of entire components such as neurons and channels while leveraging activations to identify and retain the most relevant components. Our approach is designed specifically for supervised learning tasks, where we construct a graph space that encodes the separation capabilities of each component with respect to all class pairs. By employing complementary selection principles and utilizing a clustering algorithm, ACSP ensures that the selected components maintain diverse and complementary separation capabilities, reducing redundancy and maintaining high network performance. The method automatically determines the optimal subset of components in each layer, utilizing a knee-finding algorithm to select the minimal subset that preserves performance without requiring user-defined pruning volumes. Extensive experiments on multiple architectures, including VGG-16, ResNet-50, and MobileNet-V2, across datasets like CIFAR-10, CIFAR-100, and ImageNet-1K, demonstrate that ACSP achieves competitive accuracy compared to other methods while significantly reducing computational costs. This fully automated approach not only enhances scalability but also makes ACSP especially practical for real-world deployment by eliminating the need for manually defining the pruning volume.