Feature Refinement to Improve High Resolution Image Inpainting
This addresses a specific bottleneck in image inpainting for computer vision applications, though it is incremental.
The paper tackles the problem of neural network inpainting quality degradation at high resolutions by optimizing intermediate featuremaps with a multiscale consistency loss at inference, establishing a new state-of-the-art for high-resolution inpainting.
In this paper, we address the problem of degradation in inpainting quality of neural networks operating at high resolutions. Inpainting networks are often unable to generate globally coherent structures at resolutions higher than their training set. This is partially attributed to the receptive field remaining static, despite an increase in image resolution. Although downscaling the image prior to inpainting produces coherent structure, it inherently lacks detail present at higher resolutions. To get the best of both worlds, we optimize the intermediate featuremaps of a network by minimizing a multiscale consistency loss at inference. This runtime optimization improves the inpainting results and establishes a new state-of-the-art for high resolution inpainting. Code is available at: https://github.com/geomagical/lama-with-refiner/tree/refinement.