Disentangling Racial Phenotypes: Fine-Grained Control of Race-related Facial Phenotype Characteristics
This work enables data-driven racial bias mitigation in automated facial analysis tasks by allowing analysis and synthesis of facial diversity, though it is incremental as it builds on prior GAN methods.
The paper tackles the challenge of fine-grained control over race-related facial phenotype attributes in 2D images while preserving identity, proposing a novel GAN framework that achieves state-of-the-art individual control with improved photo-realistic output.
Achieving an effective fine-grained appearance variation over 2D facial images, whilst preserving facial identity, is a challenging task due to the high complexity and entanglement of common 2D facial feature encoding spaces. Despite these challenges, such fine-grained control, by way of disentanglement is a crucial enabler for data-driven racial bias mitigation strategies across multiple automated facial analysis tasks, as it allows to analyse, characterise and synthesise human facial diversity. In this paper, we propose a novel GAN framework to enable fine-grained control over individual race-related phenotype attributes of the facial images. Our framework factors the latent (feature) space into elements that correspond to race-related facial phenotype representations, thereby separating phenotype aspects (e.g. skin, hair colour, nose, eye, mouth shapes), which are notoriously difficult to annotate robustly in real-world facial data. Concurrently, we also introduce a high quality augmented, diverse 2D face image dataset drawn from CelebA-HQ for GAN training. Unlike prior work, our framework only relies upon 2D imagery and related parameters to achieve state-of-the-art individual control over race-related phenotype attributes with improved photo-realistic output.