Pairwise-GAN: Pose-based View Synthesis through Pair-Wise Training
This work addresses a domain-specific bottleneck in computer vision for face analysis, but it is incremental as it builds on existing GAN methods like Pix2Pix and CycleGAN.
The paper tackles the problem of synthesizing frontal faces from side-pose facial images, which is crucial for applications like 3D face reconstruction and face recognition in the wild. The results show that their proposed Pairwise-GAN model achieves a 5.4% improvement in average similarity over CycleGAN and 9.1% over Pix2Pix.
Three-dimensional face reconstruction is one of the popular applications in computer vision. However, even state-of-the-art models still require frontal face as inputs, which restricts its usage scenarios in the wild. A similar dilemma also happens in face recognition. New research designed to recover the frontal face from a single side-pose facial image has emerged. The state-of-the-art in this area is the Face-Transformation generative adversarial network, which is based on the CycleGAN. This inspired our research which explores the performance of two models from pixel transformation in frontal facial synthesis, Pix2Pix and CycleGAN. We conducted the experiments on five different loss functions on Pix2Pix to improve its performance, then followed by proposing a new network Pairwise-GAN in frontal facial synthesis. Pairwise-GAN uses two parallel U-Nets as the generator and PatchGAN as the discriminator. The detailed hyper-parameters are also discussed. Based on the quantitative measurement by face similarity comparison, our results showed that Pix2Pix with L1 loss, gradient difference loss, and identity loss results in 2.72% of improvement at average similarity compared to the default Pix2Pix model. Additionally, the performance of Pairwise-GAN is 5.4% better than the CycleGAN and 9.1% than the Pix2Pix at average similarity.