Explaining Bias in Deep Face Recognition via Image Characteristics
This work addresses fairness issues in face recognition systems, which is critical for ensuring equitable deployment in security and usability applications, though it is incremental as it builds on existing evaluation frameworks.
The paper tackles the problem of understanding bias in deep face recognition models by analyzing how performance varies with both protected and non-protected image attributes, finding that trends from single-attribute analyses disappear or reverse in multi-attribute groups and that non-protected attributes also contribute to disparities.
In this paper, we propose a novel explanatory framework aimed to provide a better understanding of how face recognition models perform as the underlying data characteristics (protected attributes: gender, ethnicity, age; non-protected attributes: facial hair, makeup, accessories, face orientation and occlusion, image distortion, emotions) on which they are tested change. With our framework, we evaluate ten state-of-the-art face recognition models, comparing their fairness in terms of security and usability on two data sets, involving six groups based on gender and ethnicity. We then analyze the impact of image characteristics on models performance. Our results show that trends appearing in a single-attribute analysis disappear or reverse when multi-attribute groups are considered, and that performance disparities are also related to non-protected attributes. Source code: https://cutt.ly/2XwRLiA.