The Unfairness of $\varepsilon$-Fairness
This addresses fairness evaluation in decision-making processes for domains like admissions and credit, but it is incremental as it builds on existing fairness concepts with a new perspective.
The paper tackles the problem that probabilistic fairness metrics like ε-fairness may not capture real-world impacts, showing they can lead to maximally unfair outcomes, and proposes a utility-based approach to better measure fairness, illustrated with examples like college admissions where enhancing completion rates is crucial.
Fairness in decision-making processes is often quantified using probabilistic metrics. However, these metrics may not fully capture the real-world consequences of unfairness. In this article, we adopt a utility-based approach to more accurately measure the real-world impacts of decision-making process. In particular, we show that if the concept of $\varepsilon$-fairness is employed, it can possibly lead to outcomes that are maximally unfair in the real-world context. Additionally, we address the common issue of unavailable data on false negatives by proposing a reduced setting that still captures essential fairness considerations. We illustrate our findings with two real-world examples: college admissions and credit risk assessment. Our analysis reveals that while traditional probability-based evaluations might suggest fairness, a utility-based approach uncovers the necessary actions to truly achieve equality. For instance, in the college admission case, we find that enhancing completion rates is crucial for ensuring fairness. Summarizing, this paper highlights the importance of considering the real-world context when evaluating fairness.