How Do Fairness Definitions Fare? Examining Public Attitudes Towards Algorithmic Definitions of Fairness
This work addresses the problem of aligning algorithmic fairness definitions with public values for policymakers and developers, though it is incremental in testing existing definitions.
The study investigated public perceptions of three algorithmic fairness definitions in loan decisions, finding that calibrated fairness was most preferred and results supported affirmative action principles.
What is the best way to define algorithmic fairness? While many definitions of fairness have been proposed in the computer science literature, there is no clear agreement over a particular definition. In this work, we investigate ordinary people's perceptions of three of these fairness definitions. Across two online experiments, we test which definitions people perceive to be the fairest in the context of loan decisions, and whether fairness perceptions change with the addition of sensitive information (i.e., race of the loan applicants). Overall, one definition (calibrated fairness) tends to be more preferred than the others, and the results also provide support for the principle of affirmative action.