Semantic Photo Manipulation with a Generative Image Prior
This work addresses the problem of semantic photo manipulation for users needing precise and consistent edits in images, representing an incremental improvement over existing GAN-based methods.
The paper tackles the challenge of precisely manipulating high-level attributes in existing natural photographs using GANs by adapting the GAN's image prior to individual image statistics, enabling accurate reconstruction and consistent new content synthesis. It demonstrates effectiveness through quantitative and qualitative comparisons on tasks like object synthesis, removal, and appearance changes.
Despite the recent success of GANs in synthesizing images conditioned on inputs such as a user sketch, text, or semantic labels, manipulating the high-level attributes of an existing natural photograph with GANs is challenging for two reasons. First, it is hard for GANs to precisely reproduce an input image. Second, after manipulation, the newly synthesized pixels often do not fit the original image. In this paper, we address these issues by adapting the image prior learned by GANs to image statistics of an individual image. Our method can accurately reconstruct the input image and synthesize new content, consistent with the appearance of the input image. We demonstrate our interactive system on several semantic image editing tasks, including synthesizing new objects consistent with background, removing unwanted objects, and changing the appearance of an object. Quantitative and qualitative comparisons against several existing methods demonstrate the effectiveness of our method.