Discovering Class-Specific GAN Controls for Semantic Image Synthesis
This work addresses a gap in conditional GANs for semantic image synthesis, enabling fine-grained control over specific classes, which is incremental but useful for applications like image editing and generation.
The paper tackles the problem of discovering latent directions for conditional GANs in semantic image synthesis, which was previously unexplored, and proposes an optimization method that finds class-specific directions to control local appearance, achieving diverse and semantically meaningful edits as demonstrated through visual and quantitative evaluation.
Prior work has extensively studied the latent space structure of GANs for unconditional image synthesis, enabling global editing of generated images by the unsupervised discovery of interpretable latent directions. However, the discovery of latent directions for conditional GANs for semantic image synthesis (SIS) has remained unexplored. In this work, we specifically focus on addressing this gap. We propose a novel optimization method for finding spatially disentangled class-specific directions in the latent space of pretrained SIS models. We show that the latent directions found by our method can effectively control the local appearance of semantic classes, e.g., changing their internal structure, texture or color independently from each other. Visual inspection and quantitative evaluation of the discovered GAN controls on various datasets demonstrate that our method discovers a diverse set of unique and semantically meaningful latent directions for class-specific edits.