Machine learning fairness notions: Bridging the gap with real-world applications
It addresses the challenge of applying fairness notions in real-world applications, offering practical guidance for practitioners and policymakers, but is incremental as it builds on existing surveys by focusing on selection criteria.
This paper surveys various machine learning fairness notions and provides a decision diagram to help practitioners and policymakers select the most appropriate fairness notion for specific real-world scenarios, based on analyzing scenario characteristics and fairness notion behaviors.
Fairness emerged as an important requirement to guarantee that Machine Learning (ML) predictive systems do not discriminate against specific individuals or entire sub-populations, in particular, minorities. Given the inherent subjectivity of viewing the concept of fairness, several notions of fairness have been introduced in the literature. This paper is a survey that illustrates the subtleties between fairness notions through a large number of examples and scenarios. In addition, unlike other surveys in the literature, it addresses the question of: which notion of fairness is most suited to a given real-world scenario and why? Our attempt to answer this question consists in (1) identifying the set of fairness-related characteristics of the real-world scenario at hand, (2) analyzing the behavior of each fairness notion, and then (3) fitting these two elements to recommend the most suitable fairness notion in every specific setup. The results are summarized in a decision diagram that can be used by practitioners and policymakers to navigate the relatively large catalog of ML.