How fair can we go in machine learning? Assessing the boundaries of fairness in decision trees
This work addresses the need for stakeholders in sociotechnical systems to assess trade-offs between fairness and accuracy in decision-making, though it is incremental as it builds on existing fairness-aware approaches.
The paper tackles the problem of understanding the statistical limits of fairness in machine learning by introducing a multi-objective framework to optimize accuracy and fairness measures, providing a Pareto front of best feasible solutions for decision tree classifiers. The result shows that the method can optimize models to be fairer with only a small increase in classification error.
Fair machine learning works have been focusing on the development of equitable algorithms that address discrimination of certain groups. Yet, many of these fairness-aware approaches aim to obtain a unique solution to the problem, which leads to a poor understanding of the statistical limits of bias mitigation interventions. We present the first methodology that allows to explore those limits within a multi-objective framework that seeks to optimize any measure of accuracy and fairness and provides a Pareto front with the best feasible solutions. In this work, we focus our study on decision tree classifiers since they are widely accepted in machine learning, are easy to interpret and can deal with non-numerical information naturally. We conclude experimentally that our method can optimize decision tree models by being fairer with a small cost of the classification error. We believe that our contribution will help stakeholders of sociotechnical systems to assess how far they can go being fair and accurate, thus serving in the support of enhanced decision making where machine learning is used.