Navigating Towards Fairness with Data Selection
This addresses fairness challenges for practitioners dealing with large-scale datasets, offering a flexible solution, though it is incremental as it builds on existing fairness techniques.
The paper tackles the problem of label bias in machine learning by introducing a data selection method that efficiently mitigates bias without modifying model architectures, achieving improved fairness across diverse datasets in experiments.
Machine learning algorithms often struggle to eliminate inherent data biases, particularly those arising from unreliable labels, which poses a significant challenge in ensuring fairness. Existing fairness techniques that address label bias typically involve modifying models and intervening in the training process, but these lack flexibility for large-scale datasets. To address this limitation, we introduce a data selection method designed to efficiently and flexibly mitigate label bias, tailored to more practical needs. Our approach utilizes a zero-shot predictor as a proxy model that simulates training on a clean holdout set. This strategy, supported by peer predictions, ensures the fairness of the proxy model and eliminates the need for an additional holdout set, which is a common requirement in previous methods. Without altering the classifier's architecture, our modality-agnostic method effectively selects appropriate training data and has proven efficient and effective in handling label bias and improving fairness across diverse datasets in experimental evaluations.