Style Transformer for Image Inversion and Editing
This work addresses a specific bottleneck in image generation and editing for StyleGAN users, representing an incremental improvement over existing methods.
The paper tackles the problem of GAN inversion methods failing to simultaneously achieve reliable reconstruction and flexible editing for StyleGAN, presenting a transformer-based model that reduces distortions and improves editing quality, with experiments showing better performance on inversion and editing tasks.
Existing GAN inversion methods fail to provide latent codes for reliable reconstruction and flexible editing simultaneously. This paper presents a transformer-based image inversion and editing model for pretrained StyleGAN which is not only with less distortions, but also of high quality and flexibility for editing. The proposed model employs a CNN encoder to provide multi-scale image features as keys and values. Meanwhile it regards the style code to be determined for different layers of the generator as queries. It first initializes query tokens as learnable parameters and maps them into W+ space. Then the multi-stage alternate self- and cross-attention are utilized, updating queries with the purpose of inverting the input by the generator. Moreover, based on the inverted code, we investigate the reference- and label-based attribute editing through a pretrained latent classifier, and achieve flexible image-to-image translation with high quality results. Extensive experiments are carried out, showing better performances on both inversion and editing tasks within StyleGAN.