Ensuring Fairness Beyond the Training Data
This addresses the issue of fairness in machine learning for real-world applications where training data may not fully represent deployment conditions, though it is incremental by extending existing fairness methods to include robustness.
The paper tackles the problem of designing fair classifiers that remain fair under distributional shifts, developing a method that ensures fairness and retains accuracy across a range of perturbations on test sets, with experiments showing a trade-off between fairness robustness and accuracy.
We initiate the study of fair classifiers that are robust to perturbations in the training distribution. Despite recent progress, the literature on fairness has largely ignored the design of fair and robust classifiers. In this work, we develop classifiers that are fair not only with respect to the training distribution, but also for a class of distributions that are weighted perturbations of the training samples. We formulate a min-max objective function whose goal is to minimize a distributionally robust training loss, and at the same time, find a classifier that is fair with respect to a class of distributions. We first reduce this problem to finding a fair classifier that is robust with respect to the class of distributions. Based on online learning algorithm, we develop an iterative algorithm that provably converges to such a fair and robust solution. Experiments on standard machine learning fairness datasets suggest that, compared to the state-of-the-art fair classifiers, our classifier retains fairness guarantees and test accuracy for a large class of perturbations on the test set. Furthermore, our experiments show that there is an inherent trade-off between fairness robustness and accuracy of such classifiers.