FairCanary: Rapid Continuous Explainable Fairness
This addresses the need for rapid, explainable fairness monitoring in deployed AI systems, particularly under regulatory requirements, though it is incremental in improving efficiency over existing approaches.
The paper tackles the problem of monitoring fairness in deployed machine learning models by introducing Quantile Demographic Drift (QDD), a novel bias quantification metric that measures differences in prediction distributions over subgroups without requiring outcome labels, and integrates it into FairCanary, a system that computes feature-level bias explanations an order of magnitude faster than previous methods.
Systems that offer continuous model monitoring have emerged in response to (1) well-documented failures of deployed Machine Learning (ML) and Artificial Intelligence (AI) models and (2) new regulatory requirements impacting these models. Existing monitoring systems continuously track the performance of deployed ML models and compute feature importance (a.k.a. explanations) for each prediction to help developers identify the root causes of emergent model performance problems. We present Quantile Demographic Drift (QDD), a novel model bias quantification metric that uses quantile binning to measure differences in the overall prediction distributions over subgroups. QDD is ideal for continuous monitoring scenarios, does not suffer from the statistical limitations of conventional threshold-based bias metrics, and does not require outcome labels (which may not be available at runtime). We incorporate QDD into a continuous model monitoring system, called FairCanary, that reuses existing explanations computed for each individual prediction to quickly compute explanations for the QDD bias metrics. This optimization makes FairCanary an order of magnitude faster than previous work that has tried to generate feature-level bias explanations.