One-Shot Face Reenactment on Megapixels
This work addresses the need for high-quality face reenactment in applications like face frontalization and talking head generation, representing a strong specific gain in this domain.
The paper tackles the problem of low-resolution and unrealistic results in face reenactment by proposing MegaFR, a one-shot method that achieves high-resolution, photorealistic outputs, outperforming conventional methods in disentangling identity from expression and head pose.
The goal of face reenactment is to transfer a target expression and head pose to a source face while preserving the source identity. With the popularity of face-related applications, there has been much research on this topic. However, the results of existing methods are still limited to low-resolution and lack photorealism. In this work, we present a one-shot and high-resolution face reenactment method called MegaFR. To be precise, we leverage StyleGAN by using 3DMM-based rendering images and overcome the lack of high-quality video datasets by designing a loss function that works without high-quality videos. Also, we apply iterative refinement to deal with extreme poses and/or expressions. Since the proposed method controls source images through 3DMM parameters, we can explicitly manipulate source images. We apply MegaFR to various applications such as face frontalization, eye in-painting, and talking head generation. Experimental results show that our method successfully disentangles identity from expression and head pose, and outperforms conventional methods.