Outlier-Aware Post-Training Quantization for Image Super-Resolution
This addresses the need for efficient inference in image super-resolution by improving post-training quantization, though it is incremental as it builds on existing PTQ methods.
The paper tackles the problem of performance degradation in post-training quantization for image super-resolution due to activation outliers and varying layer sensitivities, proposing a dual-region quantization strategy and sensitivity-aware finetuning that outperforms existing PTQ methods and achieves performance comparable to quantization-aware training with at least a 75x speedup.
Quantization techniques, including quantization-aware training (QAT) and post-training quantization (PTQ), have become essential for inference acceleration of image super-resolution (SR) networks. Compared to QAT, PTQ has garnered significant attention as it eliminates the need for ground truth and model retraining. However, existing PTQ methods for SR often fail to achieve satisfactory performance as they overlook the impact of outliers in activation. Our empirical analysis reveals that these prevalent activation outliers are strongly correlated with image color information, and directly removing them leads to significant performance degradation. Motivated by this, we propose a dual-region quantization strategy that partitions activations into an outlier region and a dense region, applying uniform quantization to each region independently to better balance bit-width allocation. Furthermore, we observe that different network layers exhibit varying sensitivities to quantization, leading to different levels of performance degradation. To address this, we introduce sensitivity-aware finetuning that encourages the model to focus more on highly sensitive layers, further enhancing quantization performance. Extensive experiments demonstrate that our method outperforms existing PTQ approaches across various SR networks and datasets, while achieving performance comparable to QAT methods in most scenarios with at least a 75 speedup.