Robust Single-Image Super-Resolution via CNNs and TV-TV Minimization
This work solves the issue of deployment robustness in super-resolution for image processing applications, though it is incremental as it builds on existing CNN methods.
The paper tackles the problem of single-image super-resolution by addressing the inconsistency between CNN outputs and low-resolution inputs, proposing a TV-TV minimization post-processing method that improves image quality and robustness to operator mismatch, achieving gains in PSNR and SSIM metrics.
Single-image super-resolution is the process of increasing the resolution of an image, obtaining a high-resolution (HR) image from a low-resolution (LR) one. By leveraging large training datasets, convolutional neural networks (CNNs) currently achieve the state-of-the-art performance in this task. Yet, during testing/deployment, they fail to enforce consistency between the HR and LR images: if we downsample the output HR image, it never matches its LR input. Based on this observation, we propose to post-process the CNN outputs with an optimization problem that we call TV-TV minimization, which enforces consistency. As our extensive experiments show, such post-processing not only improves the quality of the images, in terms of PSNR and SSIM, but also makes the super-resolution task robust to operator mismatch, i.e., when the true downsampling operator is different from the one used to create the training dataset.