SRWarp: Generalized Image Super-Resolution under Arbitrary Transformation
This work addresses the problem of flexible image editing for users needing arbitrary target resolutions, representing an incremental advance over recent real-valued upsampling methods.
The paper tackles the limitation of conventional super-resolution methods that only support predetermined integer scaling factors by proposing SRWarp, a framework for generalized image super-resolution under arbitrary transformations, achieving visually favorable results across various deformations.
Deep CNNs have achieved significant successes in image processing and its applications, including single image super-resolution (SR). However, conventional methods still resort to some predetermined integer scaling factors, e.g., x2 or x4. Thus, they are difficult to be applied when arbitrary target resolutions are required. Recent approaches extend the scope to real-valued upsampling factors, even with varying aspect ratios to handle the limitation. In this paper, we propose the SRWarp framework to further generalize the SR tasks toward an arbitrary image transformation. We interpret the traditional image warping task, specifically when the input is enlarged, as a spatially-varying SR problem. We also propose several novel formulations, including the adaptive warping layer and multiscale blending, to reconstruct visually favorable results in the transformation process. Compared with previous methods, we do not constrain the SR model on a regular grid but allow numerous possible deformations for flexible and diverse image editing. Extensive experiments and ablation studies justify the necessity and demonstrate the advantage of the proposed SRWarp method under various transformations.