Towards Threshold Invariant Fair Classification
This addresses fairness issues in ML for sensitive groups like race and gender, but it is incremental as it builds on prior fairness definitions.
The paper tackles the problem of fairness in machine learning models being sensitive to decision thresholds, proposing threshold invariant fairness to ensure equitable performance across groups regardless of threshold tuning, with experimental results showing effectiveness in alleviating this sensitivity.
Effective machine learning models can automatically learn useful information from a large quantity of data and provide decisions in a high accuracy. These models may, however, lead to unfair predictions in certain sense among the population groups of interest, where the grouping is based on such sensitive attributes as race and gender. Various fairness definitions, such as demographic parity and equalized odds, were proposed in prior art to ensure that decisions guided by the machine learning models are equitable. Unfortunately, the "fair" model trained with these fairness definitions is threshold sensitive, i.e., the condition of fairness may no longer hold true when tuning the decision threshold. This paper introduces the notion of threshold invariant fairness, which enforces equitable performances across different groups independent of the decision threshold. To achieve this goal, this paper proposes to equalize the risk distributions among the groups via two approximation methods. Experimental results demonstrate that the proposed methodology is effective to alleviate the threshold sensitivity in machine learning models designed to achieve fairness.