MorphGAN: One-Shot Face Synthesis GAN for Detecting Recognition Bias
This work provides a tool for researchers and developers to create controlled test sets for detecting robustness issues and biases in face recognition systems, and to augment scarce training data for improved model performance.
This paper introduces MorphGAN, a one-shot face synthesis GAN that generates images of unseen people with controlled head pose and facial expressions. This method addresses the challenge of creating targeted datasets for bias detection in face recognition, and can also augment small datasets, improving recognition performance by up to 9%.
To detect bias in face recognition networks, it can be useful to probe a network under test using samples in which only specific attributes vary in some controlled way. However, capturing a sufficiently large dataset with specific control over the attributes of interest is difficult. In this work, we describe a simulator that applies specific head pose and facial expression adjustments to images of previously unseen people. The simulator first fits a 3D morphable model to a provided image, applies the desired head pose and facial expression controls, then renders the model into an image. Next, a conditional Generative Adversarial Network (GAN) conditioned on the original image and the rendered morphable model is used to produce the image of the original person with the new facial expression and head pose. We call this conditional GAN -- MorphGAN. Images generated using MorphGAN conserve the identity of the person in the original image, and the provided control over head pose and facial expression allows test sets to be created to identify robustness issues of a facial recognition deep network with respect to pose and expression. Images generated by MorphGAN can also serve as data augmentation when training data are scarce. We show that by augmenting small datasets of faces with new poses and expressions improves the recognition performance by up to 9% depending on the augmentation and data scarcity.