Fix the Noise: Disentangling Source Feature for Transfer Learning of StyleGAN
This work addresses a specific bottleneck in StyleGAN-based domain translation for researchers and practitioners, offering an incremental improvement over previous methods by enhancing control and quality without additional computational models.
The paper tackles the problem of limited visual quality and restricted control over source features in StyleGAN transfer learning for domain translation, proposing a feature matching loss and a disentangled subspace strategy called FixNoise that enables smooth control of source features in a single model, resulting in more consistent and realistic image generation as demonstrated in experiments.
Transfer learning of StyleGAN has recently shown great potential to solve diverse tasks, especially in domain translation. Previous methods utilized a source model by swapping or freezing weights during transfer learning, however, they have limitations on visual quality and controlling source features. In other words, they require additional models that are computationally demanding and have restricted control steps that prevent a smooth transition. In this paper, we propose a new approach to overcome these limitations. Instead of swapping or freezing, we introduce a simple feature matching loss to improve generation quality. In addition, to control the degree of source features, we train a target model with the proposed strategy, FixNoise, to preserve the source features only in a disentangled subspace of a target feature space. Owing to the disentangled feature space, our method can smoothly control the degree of the source features in a single model. Extensive experiments demonstrate that the proposed method can generate more consistent and realistic images than previous works.