Beard Segmentation and Recognition Bias
This addresses bias in face recognition systems for individuals with different facial hairstyles, particularly for African-American and Caucasian males, and is incremental as it builds on existing attribute classification methods.
The study tackled the problem of facial hair causing bias in face recognition accuracy by developing a facial hair segmentation model and using it to categorize image pairs by hairstyle similarity, finding that False Match Rate (FMR) varied by factors over 10 for African-American males and over 25 for Caucasian males. They proposed adaptive thresholding based on facial hairstyle similarity, which reduced the FMR ratio to 1.8 for African-Americans and 1.3 for Caucasians.
A person's facial hairstyle, such as presence and size of beard, can significantly impact face recognition accuracy. There are publicly-available deep networks that achieve reasonable accuracy at binary attribute classification, such as beard / no beard, but few if any that segment the facial hair region. To investigate the effect of facial hair in a rigorous manner, we first created a set of fine-grained facial hair annotations to train a segmentation model and evaluate its accuracy across African-American and Caucasian face images. We then use our facial hair segmentations to categorize image pairs according to the degree of difference or similarity in the facial hairstyle. We find that the False Match Rate (FMR) for image pairs with different categories of facial hairstyle varies by a factor of over 10 for African-American males and over 25 for Caucasian males. To reduce the bias across image pairs with different facial hairstyles, we propose a scheme for adaptive thresholding based on facial hairstyle similarity. Evaluation on a subject-disjoint set of images shows that adaptive similarity thresholding based on facial hairstyles of the image pair reduces the ratio between the highest and lowest FMR across facial hairstyle categories for African-American from 10.7 to 1.8 and for Caucasians from 25.9 to 1.3. Facial hair annotations and facial hair segmentation model will be publicly available.