Styleclone: Face Stylization with Diffusion Based Data Augmentation
This work addresses the challenge of generating high-quality stylized faces efficiently for applications in digital art and media, though it is incremental as it builds on existing diffusion and image-to-image translation techniques.
The paper tackles the problem of face stylization with limited style images by using diffusion-based data augmentation to enhance style dataset diversity, resulting in improved stylization quality, better content preservation, and significantly faster inference compared to diffusion-based methods.
We present StyleClone, a method for training image-to-image translation networks to stylize faces in a specific style, even with limited style images. Our approach leverages textual inversion and diffusion-based guided image generation to augment small style datasets. By systematically generating diverse style samples guided by both the original style images and real face images, we significantly enhance the diversity of the style dataset. Using this augmented dataset, we train fast image-to-image translation networks that outperform diffusion-based methods in speed and quality. Experiments on multiple styles demonstrate that our method improves stylization quality, better preserves source image content, and significantly accelerates inference. Additionally, we provide a systematic evaluation of the augmentation techniques and their impact on stylization performance.