Fair Active Learning in Low-Data Regimes
This work addresses fairness challenges in low-data regimes, which is critical for applications where collecting labeled data is costly, though it appears incremental as it builds on existing active learning and fairness methods.
The paper tackles the problem of ensuring fairness in machine learning models when labeled data is scarce, by introducing an active learning framework that combines posterior sampling with fair classification, resulting in improved accuracy while satisfying fairness constraints with high probability on benchmark datasets.
In critical machine learning applications, ensuring fairness is essential to avoid perpetuating social inequities. In this work, we address the challenges of reducing bias and improving accuracy in data-scarce environments, where the cost of collecting labeled data prohibits the use of large, labeled datasets. In such settings, active learning promises to maximize marginal accuracy gains of small amounts of labeled data. However, existing applications of active learning for fairness fail to deliver on this, typically requiring large labeled datasets, or failing to ensure the desired fairness tolerance is met on the population distribution. To address such limitations, we introduce an innovative active learning framework that combines an exploration procedure inspired by posterior sampling with a fair classification subroutine. We demonstrate that this framework performs effectively in very data-scarce regimes, maximizing accuracy while satisfying fairness constraints with high probability. We evaluate our proposed approach using well-established real-world benchmark datasets and compare it against state-of-the-art methods, demonstrating its effectiveness in producing fair models, and improvement over existing methods.