Smoothness Similarity Regularization for Few-Shot GAN Adaptation
This addresses the challenge of adapting GANs to small datasets with few images, particularly when domains differ structurally, which is incremental but improves robustness in few-shot learning.
The paper tackles the problem of few-shot GAN adaptation, where existing methods struggle with training instabilities or memorization when source and target domains are structurally dissimilar, and proposes a smoothness similarity regularization to transfer learned smoothness from pre-trained GANs, significantly outperforming prior methods in dissimilar domains while matching state-of-the-art in similar ones.
The task of few-shot GAN adaptation aims to adapt a pre-trained GAN model to a small dataset with very few training images. While existing methods perform well when the dataset for pre-training is structurally similar to the target dataset, the approaches suffer from training instabilities or memorization issues when the objects in the two domains have a very different structure. To mitigate this limitation, we propose a new smoothness similarity regularization that transfers the inherently learned smoothness of the pre-trained GAN to the few-shot target domain even if the two domains are very different. We evaluate our approach by adapting an unconditional and a class-conditional GAN to diverse few-shot target domains. Our proposed method significantly outperforms prior few-shot GAN adaptation methods in the challenging case of structurally dissimilar source-target domains, while performing on par with the state of the art for similar source-target domains.