Towards thinner convolutional neural networks through Gradually Global Pruning
This work addresses the need for efficient neural networks on devices with limited resources, but it is incremental as it builds on existing pruning methods with a novel global approach.
The paper tackles the problem of reducing storage and computation costs in deep neural networks for resource-limited devices by proposing a gradually global pruning scheme that removes neurons across all layers, automatically finding thinner sub-networks while maintaining performance.
Deep network pruning is an effective method to reduce the storage and computation cost of deep neural networks when applying them to resource-limited devices. Among many pruning granularities, neuron level pruning will remove redundant neurons and filters in the model and result in thinner networks. In this paper, we propose a gradually global pruning scheme for neuron level pruning. In each pruning step, a small percent of neurons were selected and dropped across all layers in the model. We also propose a simple method to eliminate the biases in evaluating the importance of neurons to make the scheme feasible. Compared with layer-wise pruning scheme, our scheme avoid the difficulty in determining the redundancy in each layer and is more effective for deep networks. Our scheme would automatically find a thinner sub-network in original network under a given performance.