Information-Theoretic Bias Assessment Of Learned Representations Of Pretrained Face Recognition
This addresses fairness issues in face recognition for minorities, though it appears incremental as it focuses on improving bias assessment metrics rather than debiasing methods.
The authors tackled the problem of bias assessment in pretrained face recognition systems by proposing an information-theoretic metric to quantify bias against protected demographic attributes, demonstrating its advantages over existing metrics through clear discrimination and small variation.
As equality issues in the use of face recognition have garnered a lot of attention lately, greater efforts have been made to debiased deep learning models to improve fairness to minorities. However, there is still no clear definition nor sufficient analysis for bias assessment metrics. We propose an information-theoretic, independent bias assessment metric to identify degree of bias against protected demographic attributes from learned representations of pretrained facial recognition systems. Our metric differs from other methods that rely on classification accuracy or examine the differences between ground truth and predicted labels of protected attributes predicted using a shallow network. Also, we argue, theoretically and experimentally, that logits-level loss is not adequate to explain bias since predictors based on neural networks will always find correlations. Further, we present a synthetic dataset that mitigates the issue of insufficient samples in certain cohorts. Lastly, we establish a benchmark metric by presenting advantages in clear discrimination and small variation comparing with other metrics, and evaluate the performance of different debiased models with the proposed metric.