Attentive Fine-Grained Structured Sparsity for Image Restoration
This work addresses the problem of high computational costs in image restoration for practical applications, offering an incremental optimization to existing pruning techniques.
The paper tackles the computational inefficiency of large deep models in image restoration by proposing a novel pruning method that determines layer-specific pruning ratios for N:M structured sparsity, achieving significant performance improvements over previous pruning methods in super-resolution and deblurring tasks.
Image restoration tasks have witnessed great performance improvement in recent years by developing large deep models. Despite the outstanding performance, the heavy computation demanded by the deep models has restricted the application of image restoration. To lift the restriction, it is required to reduce the size of the networks while maintaining accuracy. Recently, N:M structured pruning has appeared as one of the effective and practical pruning approaches for making the model efficient with the accuracy constraint. However, it fails to account for different computational complexities and performance requirements for different layers of an image restoration network. To further optimize the trade-off between the efficiency and the restoration accuracy, we propose a novel pruning method that determines the pruning ratio for N:M structured sparsity at each layer. Extensive experimental results on super-resolution and deblurring tasks demonstrate the efficacy of our method which outperforms previous pruning methods significantly. PyTorch implementation for the proposed methods is available at https://github.com/JungHunOh/SLS_CVPR2022.