DRNet: Dissect and Reconstruct the Convolutional Neural Network via Interpretable Manners
This addresses efficiency and accuracy issues for users applying pre-trained ConvNets to specific sub-tasks, though it is incremental as it builds on existing network pruning and interpretability techniques.
The paper tackles the redundancy of convolutional neural network channels by proposing an interpretable method to identify important channels per class and dynamically run only necessary ones for sub-tasks, achieving 11% parameter usage with negligible accuracy loss on CIFAR-10 and a 14.29% accuracy gain on ImageNet for two-class sub-tasks.
Convolutional neural networks (ConvNets) are widely used in real life. People usually use ConvNets which pre-trained on a fixed number of classes. However, for different application scenarios, we usually do not need all of the classes, which means ConvNets are redundant when dealing with these tasks. This paper focuses on the redundancy of ConvNet channels. We proposed a novel idea: using an interpretable manner to find the most important channels for every single class (dissect), and dynamically run channels according to classes in need (reconstruct). For VGG16 pre-trained on CIFAR-10, we only run 11\% parameters for two-classes sub-tasks on average with negligible accuracy loss. For VGG16 pre-trained on ImageNet, our method averagely gains 14.29\% accuracy promotion for two-classes sub-tasks. In addition, analysis show that our method captures some semantic meanings of channels, and uses the context information more targeted for sub-tasks of ConvNets.