Residual Local Feature Network for Efficient Super-Resolution
This work addresses efficient super-resolution deployment on resource-constrained devices, representing an incremental improvement with specific optimizations.
The authors tackled the problem of efficient single image super-resolution for resource-constrained devices by proposing the Residual Local Feature Network (RLFN), which uses three convolutional layers for residual local feature learning to simplify feature aggregation, achieving state-of-the-art runtime performance while maintaining PSNR and SSIM metrics and winning first place in the NTIRE 2022 challenge.
Deep learning based approaches has achieved great performance in single image super-resolution (SISR). However, recent advances in efficient super-resolution focus on reducing the number of parameters and FLOPs, and they aggregate more powerful features by improving feature utilization through complex layer connection strategies. These structures may not be necessary to achieve higher running speed, which makes them difficult to be deployed to resource-constrained devices. In this work, we propose a novel Residual Local Feature Network (RLFN). The main idea is using three convolutional layers for residual local feature learning to simplify feature aggregation, which achieves a good trade-off between model performance and inference time. Moreover, we revisit the popular contrastive loss and observe that the selection of intermediate features of its feature extractor has great influence on the performance. Besides, we propose a novel multi-stage warm-start training strategy. In each stage, the pre-trained weights from previous stages are utilized to improve the model performance. Combined with the improved contrastive loss and training strategy, the proposed RLFN outperforms all the state-of-the-art efficient image SR models in terms of runtime while maintaining both PSNR and SSIM for SR. In addition, we won the first place in the runtime track of the NTIRE 2022 efficient super-resolution challenge. Code will be available at https://github.com/fyan111/RLFN.