Inference for an Algorithmic Fairness-Accuracy Frontier
This work addresses fairness-accuracy trade-offs in high-stakes decision-making for researchers and practitioners, offering a method to evaluate and improve algorithms, though it is incremental as it builds on an existing frontier concept.
The paper tackles the trade-off between fairness and accuracy in algorithmic decision-making by proposing a debiased machine learning estimator for the fairness-accuracy frontier, deriving its asymptotic distribution and inference methods to test hypotheses like optimal exclusion of group identity and existence of less discriminatory alternatives, with application showing improvements in hospital care management algorithms.
Algorithms are increasingly used to aid with high-stakes decision making. Yet, their predictive ability frequently exhibits systematic variation across population subgroups. To assess the trade-off between fairness and accuracy using finite data, we propose a debiased machine learning estimator for the fairness-accuracy frontier introduced by Liang, Lu, Mu, and Okumura (2024). We derive its asymptotic distribution and propose inference methods to test key hypotheses in the fairness literature, such as (i) whether excluding group identity from use in training the algorithm is optimal and (ii) whether there are less discriminatory alternatives to a given algorithm. In addition, we construct an estimator for the distance between a given algorithm and the fairest point on the frontier, and characterize its asymptotic distribution. Using Monte Carlo simulations, we evaluate the finite-sample performance of our inference methods. We apply our framework to re-evaluate algorithms used in hospital care management and show that our approach yields alternative algorithms that lie on the fairness-accuracy frontier, offering improvements along both dimensions.