Face Translation between Images and Videos using Identity-aware CycleGAN
This work addresses facial video prediction and enhancement for applications in computer vision, but it is incremental as it generalizes existing methods to a new context.
This paper tackled the problem of unpaired face translation between images and videos, addressing challenges in robust translation and identity preservation, and demonstrated effectiveness through experiments on standard datasets with qualitative and quantitative evaluations.
This paper presents a new problem of unpaired face translation between images and videos, which can be applied to facial video prediction and enhancement. In this problem there exist two major technical challenges: 1) designing a robust translation model between static images and dynamic videos, and 2) preserving facial identity during image-video translation. To address such two problems, we generalize the state-of-the-art image-to-image translation network (Cycle-Consistent Adversarial Networks) to the image-to-video/video-to-image translation context by exploiting a image-video translation model and an identity preservation model. In particular, we apply the state-of-the-art Wasserstein GAN technique to the setting of image-video translation for better convergence, and we meanwhile introduce a face verificator to ensure the identity. Experiments on standard image/video face datasets demonstrate the effectiveness of the proposed model in both terms of qualitative and quantitative evaluations.