Transferring GANs: generating images from limited data
This addresses the challenge of data scarcity in image generation for AI applications, though it is incremental as it applies existing finetuning practices to generative models.
The paper tackles the problem of generating images with limited target data by studying domain adaptation for generative adversarial networks (GANs), showing that using knowledge from pretrained networks can shorten convergence time and significantly improve image quality, especially with limited data.
Transferring the knowledge of pretrained networks to new domains by means of finetuning is a widely used practice for applications based on discriminative models. To the best of our knowledge this practice has not been studied within the context of generative deep networks. Therefore, we study domain adaptation applied to image generation with generative adversarial networks. We evaluate several aspects of domain adaptation, including the impact of target domain size, the relative distance between source and target domain, and the initialization of conditional GANs. Our results show that using knowledge from pretrained networks can shorten the convergence time and can significantly improve the quality of the generated images, especially when the target data is limited. We show that these conclusions can also be drawn for conditional GANs even when the pretrained model was trained without conditioning. Our results also suggest that density may be more important than diversity and a dataset with one or few densely sampled classes may be a better source model than more diverse datasets such as ImageNet or Places.