Pixel-wise Dense Detector for Image Inpainting
This work improves image inpainting for computer vision applications by automating loss balancing and enhancing artifact localization, though it is incremental as it builds on existing GAN approaches.
The paper tackles the problem of image inpainting by addressing the loss of position information in artifacts and manual tuning of loss weights in GAN-based methods, proposing a detection-based generative framework that achieves superior performance on multiple public datasets.
Recent GAN-based image inpainting approaches adopt an average strategy to discriminate the generated image and output a scalar, which inevitably lose the position information of visual artifacts. Moreover, the adversarial loss and reconstruction loss (e.g., l1 loss) are combined with tradeoff weights, which are also difficult to tune. In this paper, we propose a novel detection-based generative framework for image inpainting, which adopts the min-max strategy in an adversarial process. The generator follows an encoder-decoder architecture to fill the missing regions, and the detector using weakly supervised learning localizes the position of artifacts in a pixel-wise manner. Such position information makes the generator pay attention to artifacts and further enhance them. More importantly, we explicitly insert the output of the detector into the reconstruction loss with a weighting criterion, which balances the weight of the adversarial loss and reconstruction loss automatically rather than manual operation. Experiments on multiple public datasets show the superior performance of the proposed framework. The source code is available at https://github.com/Evergrow/GDN_Inpainting.