Robustness Disparities in Face Detection
This addresses fairness and robustness problems in face detection for marginalized groups, highlighting downstream impacts on facial analysis systems, and is incremental as it extends prior work on facial analysis audits to a new component.
The paper tackles the understudied fairness issue of face detection systems by benchmarking their robustness to noise across commercial and academic models, finding that individuals who are masculine presenting, older, of darker skin type, or in dim lighting are more susceptible to errors, with specific disparities quantified in the datasets.
Facial analysis systems have been deployed by large companies and critiqued by scholars and activists for the past decade. Many existing algorithmic audits examine the performance of these systems on later stage elements of facial analysis systems like facial recognition and age, emotion, or perceived gender prediction; however, a core component to these systems has been vastly understudied from a fairness perspective: face detection, sometimes called face localization. Since face detection is a pre-requisite step in facial analysis systems, the bias we observe in face detection will flow downstream to the other components like facial recognition and emotion prediction. Additionally, no prior work has focused on the robustness of these systems under various perturbations and corruptions, which leaves open the question of how various people are impacted by these phenomena. We present the first of its kind detailed benchmark of face detection systems, specifically examining the robustness to noise of commercial and academic models. We use both standard and recently released academic facial datasets to quantitatively analyze trends in face detection robustness. Across all the datasets and systems, we generally find that photos of individuals who are $\textit{masculine presenting}$, $\textit{older}$, of $\textit{darker skin type}$, or have $\textit{dim lighting}$ are more susceptible to errors than their counterparts in other identities.