PASS: Protected Attribute Suppression System for Mitigating Bias in Face Recognition
This addresses bias and privacy issues in face recognition systems, offering a practical solution without requiring end-to-end retraining, though it is incremental as it builds on existing networks.
The paper tackles bias in face recognition by introducing PASS, a system that reduces encoding of sensitive attributes like gender and skintone in descriptors from pre-trained networks, achieving improved bias mitigation on the IJB-C dataset while maintaining high verification accuracy.
Face recognition networks encode information about sensitive attributes while being trained for identity classification. Such encoding has two major issues: (a) it makes the face representations susceptible to privacy leakage (b) it appears to contribute to bias in face recognition. However, existing bias mitigation approaches generally require end-to-end training and are unable to achieve high verification accuracy. Therefore, we present a descriptor-based adversarial de-biasing approach called `Protected Attribute Suppression System (PASS)'. PASS can be trained on top of descriptors obtained from any previously trained high-performing network to classify identities and simultaneously reduce encoding of sensitive attributes. This eliminates the need for end-to-end training. As a component of PASS, we present a novel discriminator training strategy that discourages a network from encoding protected attribute information. We show the efficacy of PASS to reduce gender and skintone information in descriptors from SOTA face recognition networks like Arcface. As a result, PASS descriptors outperform existing baselines in reducing gender and skintone bias on the IJB-C dataset, while maintaining a high verification accuracy.