Leveraging Labeled and Unlabeled Data for Consistent Fair Binary Classification
This work addresses fairness in classification for sensitive groups, but it is incremental as it builds on existing Equal Opportunity frameworks with a new calibration approach.
The authors tackled fair binary classification under Equal Opportunity by deriving that the fair optimal classifier is a recalibrated Bayes classifier with group-dependent thresholds, and they proposed a plug-in method using both labeled and unlabeled data that is statistically consistent and competitive with state-of-the-art methods on benchmark datasets.
We study the problem of fair binary classification using the notion of Equal Opportunity. It requires the true positive rate to distribute equally across the sensitive groups. Within this setting we show that the fair optimal classifier is obtained by recalibrating the Bayes classifier by a group-dependent threshold. We provide a constructive expression for the threshold. This result motivates us to devise a plug-in classification procedure based on both unlabeled and labeled datasets. While the latter is used to learn the output conditional probability, the former is used for calibration. The overall procedure can be computed in polynomial time and it is shown to be statistically consistent both in terms of the classification error and fairness measure. Finally, we present numerical experiments which indicate that our method is often superior or competitive with the state-of-the-art methods on benchmark datasets.