Benefiting from Multitask Learning to Improve Single Image Super-Resolution
This addresses the challenge of realistic texture reconstruction in super-resolution for applications like image enhancement, though it is incremental by building on existing CNN methods.
The paper tackles the problem of reconstructing fine and natural textures in single image super-resolution by introducing a decoder architecture that uses multitask learning with semantic segmentation to improve image quality, and it outperforms state-of-the-art methods in perceptual experiments on the COCO-Stuff dataset.
Despite significant progress toward super resolving more realistic images by deeper convolutional neural networks (CNNs), reconstructing fine and natural textures still remains a challenging problem. Recent works on single image super resolution (SISR) are mostly based on optimizing pixel and content wise similarity between recovered and high-resolution (HR) images and do not benefit from recognizability of semantic classes. In this paper, we introduce a novel approach using categorical information to tackle the SISR problem; we present a decoder architecture able to extract and use semantic information to super-resolve a given image by using multitask learning, simultaneously for image super-resolution and semantic segmentation. To explore categorical information during training, the proposed decoder only employs one shared deep network for two task-specific output layers. At run-time only layers resulting HR image are used and no segmentation label is required. Extensive perceptual experiments and a user study on images randomly selected from COCO-Stuff dataset demonstrate the effectiveness of our proposed method and it outperforms the state-of-the-art methods.