High-fidelity GAN Inversion with Padding Space
This work addresses a bottleneck in image editing for users of pre-trained GANs, offering more flexible manipulation, though it is incremental as it builds on existing GAN inversion techniques.
The paper tackles the problem of insufficient spatial detail recovery in GAN inversion for image editing by proposing to use the padding space of the generator alongside the latent space, achieving improved inversion quality both qualitatively and quantitatively over existing methods.
Inverting a Generative Adversarial Network (GAN) facilitates a wide range of image editing tasks using pre-trained generators. Existing methods typically employ the latent space of GANs as the inversion space yet observe the insufficient recovery of spatial details. In this work, we propose to involve the padding space of the generator to complement the latent space with spatial information. Concretely, we replace the constant padding (e.g., usually zeros) used in convolution layers with some instance-aware coefficients. In this way, the inductive bias assumed in the pre-trained model can be appropriately adapted to fit each individual image. Through learning a carefully designed encoder, we manage to improve the inversion quality both qualitatively and quantitatively, outperforming existing alternatives. We then demonstrate that such a space extension barely affects the native GAN manifold, hence we can still reuse the prior knowledge learned by GANs for various downstream applications. Beyond the editing tasks explored in prior arts, our approach allows a more flexible image manipulation, such as the separate control of face contour and facial details, and enables a novel editing manner where users can customize their own manipulations highly efficiently.