Jointly Sparse Convolutional Neural Networks in Dual Spatial-Winograd Domains
This work addresses the need for efficient CNN deployment on varied hardware systems, offering a universal compressed model without re-training, though it is incremental in combining sparsity techniques for dual domains.
The paper tackles the problem of optimizing deep convolutional neural networks for reduced complexity across different deployment platforms, achieving compressed models for ResNet-18 and AlexNet on ImageNet with compression ratios of 24.2x and 47.7x and computational cost reductions of 4.5x and 5.1x, respectively.
We consider the optimization of deep convolutional neural networks (CNNs) such that they provide good performance while having reduced complexity if deployed on either conventional systems with spatial-domain convolution or lower-complexity systems designed for Winograd convolution. The proposed framework produces one compressed model whose convolutional filters can be made sparse either in the spatial domain or in the Winograd domain. Hence, the compressed model can be deployed universally on any platform, without need for re-training on the deployed platform. To get a better compression ratio, the sparse model is compressed in the spatial domain that has a fewer number of parameters. From our experiments, we obtain $24.2\times$ and $47.7\times$ compressed models for ResNet-18 and AlexNet trained on the ImageNet dataset, while their computational cost is also reduced by $4.5\times$ and $5.1\times$, respectively.