Towards Fair Machine Learning Software: Understanding and Addressing Model Bias Through Counterfactual Thinking
This work addresses fairness issues in ML software for developers and users, offering a solution that balances fairness and performance without significant trade-offs, though it appears incremental as it builds on existing counterfactual and optimization methods.
The paper tackles the problem of fairness bugs in machine learning software, which often require sacrificing performance, by introducing a counterfactual approach that addresses root causes of bias and combines models optimized for both fairness and performance. The method significantly improves fairness while maintaining competitive performance, outperforming state-of-the-art solutions in 84.6% of cases across evaluations on 10 benchmark tasks and 8 real-world datasets.
The increasing use of Machine Learning (ML) software can lead to unfair and unethical decisions, thus fairness bugs in software are becoming a growing concern. Addressing these fairness bugs often involves sacrificing ML performance, such as accuracy. To address this issue, we present a novel counterfactual approach that uses counterfactual thinking to tackle the root causes of bias in ML software. In addition, our approach combines models optimized for both performance and fairness, resulting in an optimal solution in both aspects. We conducted a thorough evaluation of our approach on 10 benchmark tasks using a combination of 5 performance metrics, 3 fairness metrics, and 15 measurement scenarios, all applied to 8 real-world datasets. The conducted extensive evaluations show that the proposed method significantly improves the fairness of ML software while maintaining competitive performance, outperforming state-of-the-art solutions in 84.6% of overall cases based on a recent benchmarking tool.