Benchmarking Stochastic Approximation Algorithms for Fairness-Constrained Training of Deep Neural Networks
This work addresses the problem of improving fairness in machine learning models for real-world applications, but it is incremental as it benchmarks existing algorithms rather than introducing new ones.
The paper tackled the lack of standard methods for training deep neural networks with fairness constraints by creating a benchmark based on the US Census (Folktables) and comparing three unimplemented stochastic approximation algorithms, showing improvements in optimization and fairness.
The ability to train Deep Neural Networks (DNNs) with constraints is instrumental in improving the fairness of modern machine-learning models. Many algorithms have been analysed in recent years, and yet there is no standard, widely accepted method for the constrained training of DNNs. In this paper, we provide a challenging benchmark of real-world large-scale fairness-constrained learning tasks, built on top of the US Census (Folktables). We point out the theoretical challenges of such tasks and review the main approaches in stochastic approximation algorithms. Finally, we demonstrate the use of the benchmark by implementing and comparing three recently proposed, but as-of-yet unimplemented, algorithms both in terms of optimization performance, and fairness improvement. We release the code of the benchmark as a Python package at https://github.com/humancompatible/train.