ReenactGAN: Learning to Reenact Faces via Boundary Transfer
This addresses the problem of realistic and efficient face animation for applications like video editing or virtual avatars, representing a novel method for a known bottleneck.
The authors tackled face reenactment by transferring facial movements from a source to a target person using a boundary-based method, achieving photo-realistic results and real-time performance at 30 FPS on a GTX 1080 GPU.
We present a novel learning-based framework for face reenactment. The proposed method, known as ReenactGAN, is capable of transferring facial movements and expressions from monocular video input of an arbitrary person to a target person. Instead of performing a direct transfer in the pixel space, which could result in structural artifacts, we first map the source face onto a boundary latent space. A transformer is subsequently used to adapt the boundary of source face to the boundary of target face. Finally, a target-specific decoder is used to generate the reenacted target face. Thanks to the effective and reliable boundary-based transfer, our method can perform photo-realistic face reenactment. In addition, ReenactGAN is appealing in that the whole reenactment process is purely feed-forward, and thus the reenactment process can run in real-time (30 FPS on one GTX 1080 GPU). Dataset and model will be publicly available at https://wywu.github.io/projects/ReenactGAN/ReenactGAN.html