Group-based Fair Learning Leads to Counter-intuitive Predictions
This work addresses a foundational issue in fairness-aware machine learning, revealing unintended consequences that could affect individuals' understanding of predictions, though it is incremental in proposing a theoretical fix.
The paper tackles the problem of group fairness in machine learning by proposing a desirable property called slack-consistency, which requires predictions to be monotonic with respect to allowed fairness violations, and finds that standard fairness methods violate this property, leading to counter-intuitive and unintended behaviors.
A number of machine learning (ML) methods have been proposed recently to maximize model predictive accuracy while enforcing notions of group parity or fairness across sub-populations. We propose a desirable property for these procedures, slack-consistency: For any individual, the predictions of the model should be monotonic with respect to allowed slack (i.e., maximum allowed group-parity violation). Such monotonicity can be useful for individuals to understand the impact of enforcing fairness on their predictions. Surprisingly, we find that standard ML methods for enforcing fairness violate this basic property. Moreover, this undesirable behavior arises in situations agnostic to the complexity of the underlying model or approximate optimizations, suggesting that the simple act of incorporating a constraint can lead to drastically unintended behavior in ML. We present a simple theoretical method for enforcing slack-consistency, while encouraging further discussions on the unintended behaviors potentially induced when enforcing group-based parity.