Navigating Fairness Measures and Trade-Offs
This work addresses fairness trade-offs in AI for practitioners and policymakers, offering a substantive theory to guide decisions, though it is incremental in bridging philosophical justice with fairness literature.
The paper tackles the problem of bias in AI systems by addressing the trade-offs between multiple fairness measures and accuracy, proposing a principled approach based on Rawls' justice as fairness to focus on vulnerable groups and impactful fairness measures.
In order to monitor and prevent bias in AI systems we can use a wide range of (statistical) fairness measures. However, it is mathematically impossible to optimize for all of these measures at the same time. In addition, optimizing a fairness measure often greatly reduces the accuracy of the system (Kozodoi et al, 2022). As a result, we need a substantive theory that informs us how to make these decisions and for what reasons. I show that by using Rawls' notion of justice as fairness, we can create a basis for navigating fairness measures and the accuracy trade-off. In particular, this leads to a principled choice focusing on both the most vulnerable groups and the type of fairness measure that has the biggest impact on that group. This also helps to close part of the gap between philosophical accounts of distributive justice and the fairness literature that has been observed (Kuppler et al, 2021) and to operationalise the value of fairness.