Adaptive Calibration for Fair and Performant Facial Recognition
For facial recognition systems, AC provides a practical, fair calibration method that improves performance and fairness simultaneously without requiring sensitive demographic data.
Adaptive Calibration (AC) improves facial recognition by mapping cosine similarity to well-calibrated probabilities using local context, achieving better accuracy and fairness without demographic metadata, dominating existing methods across benchmarks.
We introduce Adaptive Calibration (AC), a novel calibration strategy for facial recognition that maps cosine similarity between normalized embeddings to well-calibrated probabilities. By incorporating local context into calibration, Adaptive Calibration corrects for a fundamental mismatch in cosine similarity, whereby the same distance can correspond to different match probabilities in different embedding regions. Our approach improves both overall performance and results in a fairer calibration without requiring demographic metadata. Our approach consistently dominates existing methods both on accuracy and fairness metrics across a variety of pretrained models and standard benchmarks. AC provides a practical solution for equitable facial recognition, without requiring demographic group annotations, and while improving overall performance. Unlike existing approaches, our method provides continuous, region-specific calibration that avoids "leveling down" where fairness comes at the cost of degraded performance for some groups.