Pose and Facial Expression Transfer by using StyleGAN
This work addresses the need for controllable face synthesis in computer vision, but it is incremental as it builds on existing StyleGAN2 techniques.
The authors tackled the problem of transferring pose and facial expression between face images, achieving a method that synthesizes output images with transferred attributes from source to target identities using StyleGAN2, with close-to-real-time performance.
We propose a method to transfer pose and expression between face images. Given a source and target face portrait, the model produces an output image in which the pose and expression of the source face image are transferred onto the target identity. The architecture consists of two encoders and a mapping network that projects the two inputs into the latent space of StyleGAN2, which finally generates the output. The training is self-supervised from video sequences of many individuals. Manual labeling is not required. Our model enables the synthesis of random identities with controllable pose and expression. Close-to-real-time performance is achieved.