Pixel-based Facial Expression Synthesis
This addresses the issue of poor generalization in GAN-based facial expression synthesis for deployment on resource-constrained devices, though it is incremental as it builds on localized region synthesis methods.
The paper tackles the problem of facial expression synthesis by proposing a pixel-based method that observes only one input pixel per output pixel, achieving comparable performance to state-of-the-art GANs on in-dataset images and significantly better results on out-of-dataset images with a model two orders of magnitude smaller.
Facial expression synthesis has achieved remarkable advances with the advent of Generative Adversarial Networks (GANs). However, GAN-based approaches mostly generate photo-realistic results as long as the testing data distribution is close to the training data distribution. The quality of GAN results significantly degrades when testing images are from a slightly different distribution. Moreover, recent work has shown that facial expressions can be synthesized by changing localized face regions. In this work, we propose a pixel-based facial expression synthesis method in which each output pixel observes only one input pixel. The proposed method achieves good generalization capability by leveraging only a few hundred training images. Experimental results demonstrate that the proposed method performs comparably well against state-of-the-art GANs on in-dataset images and significantly better on out-of-dataset images. In addition, the proposed model is two orders of magnitude smaller which makes it suitable for deployment on resource-constrained devices.