Empirical Likelihood-Based Fairness Auditing: Distribution-Free Certification and Flagging
This work addresses fairness auditing for high-stakes applications like recidivism prediction, offering a non-parametric method that is more efficient and robust than existing techniques, though it is incremental in improving computational efficiency and statistical validity.
The paper tackles the problem of algorithmic bias in high-stakes machine learning models by proposing a novel empirical likelihood-based framework for fairness auditing, which achieves coverage rates closer to nominal levels and reduces computational latency by several orders of magnitude compared to bootstrap-based approaches.
Machine learning models in high-stakes applications, such as recidivism prediction and automated personnel selection, often exhibit systematic performance disparities across sensitive subpopulations, raising critical concerns regarding algorithmic bias. Fairness auditing addresses these risks through two primary functions: certification, which verifies adherence to fairness constraints; and flagging, which isolates specific demographic groups experiencing disparate treatment. However, existing auditing techniques are frequently limited by restrictive distributional assumptions or prohibitive computational overhead. We propose a novel empirical likelihood-based (EL) framework that constructs robust statistical measures for model performance disparities. Unlike traditional methods, our approach is non-parametric; the proposed disparity statistics follow asymptotically chi-square or mixed chi-square distributions, ensuring valid inference without assuming underlying data distributions. This framework uses a constrained optimization profile that admits stable numerical solutions, facilitating both large-scale certification and efficient subpopulation discovery. Empirically, the EL methods outperform bootstrap-based approaches, yielding coverage rates closer to nominal levels while reducing computational latency by several orders of magnitude. We demonstrate the practical utility of this framework on the COMPAS dataset, where it successfully flags intersectional biases, specifically identifying a significantly higher positive prediction rate for African-American males under 25 and a systemic under-prediction for Caucasian females relative to the population mean.