Structured Sparsification with Joint Optimization of Group Convolution and Channel Shuffle
This work addresses network compression for efficient deployment in resource-constrained environments, but it is incremental as it builds on existing sparsification and group convolution techniques.
The paper tackles the problem of high computational cost in convolutional neural networks by proposing a structured sparsification method that automatically induces sparsity and uses group convolution with a learnable channel shuffle, achieving competitive performance with minimal accuracy drop.
Recent advances in convolutional neural networks(CNNs) usually come with the expense of excessive computational overhead and memory footprint. Network compression aims to alleviate this issue by training compact models with comparable performance. However, existing compression techniques either entail dedicated expert design or compromise with a moderate performance drop. In this paper, we propose a novel structured sparsification method for efficient network compression. The proposed method automatically induces structured sparsity on the convolutional weights, thereby facilitating the implementation of the compressed model with the highly-optimized group convolution. We further address the problem of inter-group communication with a learnable channel shuffle mechanism. The proposed approach can be easily applied to compress many network architectures with a negligible performance drop. Extensive experimental results and analysis demonstrate that our approach gives a competitive performance against the recent network compression counterparts with a sound accuracy-complexity trade-off.