Mitigating Bias in Facial Analysis Systems by Incorporating Label Diversity
This work addresses bias in facial classifiers, which is a critical issue for individuals and society, though it appears incremental as it builds on existing annotation methods.
The paper tackled algorithmic discrimination in facial analysis models by introducing a learning method that combines subjective human-based labels and objective annotations to mitigate unintended biases, achieving significant accuracy on downstream tasks.
Facial analysis models are increasingly applied in real-world applications that have significant impact on peoples' lives. However, as literature has shown, models that automatically classify facial attributes might exhibit algorithmic discrimination behavior with respect to protected groups, potentially posing negative impacts on individuals and society. It is therefore critical to develop techniques that can mitigate unintended biases in facial classifiers. Hence, in this work, we introduce a novel learning method that combines both subjective human-based labels and objective annotations based on mathematical definitions of facial traits. Specifically, we generate new objective annotations from two large-scale human-annotated dataset, each capturing a different perspective of the analyzed facial trait. We then propose an ensemble learning method, which combines individual models trained on different types of annotations. We provide an in-depth analysis of the annotation procedure as well as the datasets distribution. Moreover, we empirically demonstrate that, by incorporating label diversity, our method successfully mitigates unintended biases, while maintaining significant accuracy on the downstream tasks.