The Statistical Fairness-Accuracy Frontier
This work addresses the challenge for policymakers and practitioners in designing fair algorithms with limited data, representing an incremental advance by extending theoretical fairness-accuracy frontiers to finite-sample regimes.
The paper tackles the problem of balancing fairness and accuracy in machine learning models with limited data, deriving minimax-optimal estimators and showing how finite-sample effects asymmetrically impact group risks, with results enabling practical use of the fairness-accuracy frontier.
Machine learning models must balance accuracy and fairness, but these goals often conflict, particularly when data come from multiple demographic groups. A useful tool for understanding this trade-off is the fairness-accuracy (FA) frontier, which characterizes the set of models that cannot be simultaneously improved in both fairness and accuracy. Prior analyses of the FA frontier provide a full characterization under the assumption of complete knowledge of population distributions -- an unrealistic ideal. We study the FA frontier in the finite-sample regime, showing how it deviates from its population counterpart and quantifying the worst-case gap between them. In particular, we derive minimax-optimal estimators that depend on the designer's knowledge of the covariate distribution. For each estimator, we characterize how finite-sample effects asymmetrically impact each group's risk, and identify optimal sample allocation strategies. Our results transform the FA frontier from a theoretical construct into a practical tool for policymakers and practitioners who must often design algorithms with limited data.