Removing biased data to improve fairness and accuracy
This work is significant for practitioners and researchers aiming to mitigate bias and improve fairness in machine learning systems, particularly when dealing with historically biased datasets.
This paper addresses the problem of bias in machine learning models trained on historical data by proposing a black-box method to identify and remove biased training data. Models trained on this debiased subset achieve individual discrimination of often 0%, along with improved accuracy and reduced statistical disparity compared to models trained on the original full dataset.
Machine learning systems are often trained using data collected from historical decisions. If past decisions were biased, then automated systems that learn from historical data will also be biased. We propose a black-box approach to identify and remove biased training data. Machine learning models trained on such debiased data (a subset of the original training data) have low individual discrimination, often 0%. These models also have greater accuracy and lower statistical disparity than models trained on the full historical data. We evaluated our methodology in experiments using 6 real-world datasets. Our approach outperformed seven previous approaches in terms of individual discrimination and accuracy.