StyleGAN Salon: Multi-View Latent Optimization for Pose-Invariant Hairstyle Transfer
This work solves the problem of realistic virtual hair try-on for users in applications like fashion or entertainment, but it is incremental as it builds on existing StyleGAN-based methods.
The paper tackles the problem of transferring hairstyles from a reference image to an input photo for virtual hair try-on, addressing challenges like pose variations and occlusions, and reports outperforming prior work in a user study with more challenging scenarios.
Our paper seeks to transfer the hairstyle of a reference image to an input photo for virtual hair try-on. We target a variety of challenges scenarios, such as transforming a long hairstyle with bangs to a pixie cut, which requires removing the existing hair and inferring how the forehead would look, or transferring partially visible hair from a hat-wearing person in a different pose. Past solutions leverage StyleGAN for hallucinating any missing parts and producing a seamless face-hair composite through so-called GAN inversion or projection. However, there remains a challenge in controlling the hallucinations to accurately transfer hairstyle and preserve the face shape and identity of the input. To overcome this, we propose a multi-view optimization framework that uses "two different views" of reference composites to semantically guide occluded or ambiguous regions. Our optimization shares information between two poses, which allows us to produce high fidelity and realistic results from incomplete references. Our framework produces high-quality results and outperforms prior work in a user study that consists of significantly more challenging hair transfer scenarios than previously studied. Project page: https://stylegan-salon.github.io/.