The FairCeptron: A Framework for Measuring Human Perceptions of Algorithmic Fairness
This work addresses the critical problem of incorporating diverse human perceptions of fairness into algorithmic design, which is crucial for developers and policymakers aiming to build more equitable AI systems. It is an incremental step towards a more human-centered approach to fairness.
This paper introduces the FairCeptron framework, designed to measure human perceptions of algorithmic fairness, which often vary across demographics. The framework facilitates studying these perceptions and comparing them with existing algorithmic fairness metrics, demonstrated through a hypothetical university admission scenario involving minorities.
Measures of algorithmic fairness often do not account for human perceptions of fairness that can substantially vary between different sociodemographics and stakeholders. The FairCeptron framework is an approach for studying perceptions of fairness in algorithmic decision making such as in ranking or classification. It supports (i) studying human perceptions of fairness and (ii) comparing these human perceptions with measures of algorithmic fairness. The framework includes fairness scenario generation, fairness perception elicitation and fairness perception analysis. We demonstrate the FairCeptron framework by applying it to a hypothetical university admission context where we collect human perceptions of fairness in the presence of minorities. An implementation of the FairCeptron framework is openly available, and it can easily be adapted to study perceptions of algorithmic fairness in other application contexts. We hope our work paves the way towards elevating the role of studies of human fairness perceptions in the process of designing algorithmic decision making systems.