On the Relationship Between Explanations, Fairness Perceptions, and Decisions
This work addresses the challenge of ensuring human-AI complementarity for fairness in decision-making, but it is incremental as it synthesizes existing literature rather than introducing new empirical results.
The paper tackles the problem of whether explanations in AI systems actually help humans make fairer decisions by providing a conceptual framework to analyze the relationships between explanations, fairness perceptions, reliance, and distributive fairness, and applies it to reconcile contradictory research findings.
It is known that recommendations of AI-based systems can be incorrect or unfair. Hence, it is often proposed that a human be the final decision-maker. Prior work has argued that explanations are an essential pathway to help human decision-makers enhance decision quality and mitigate bias, i.e., facilitate human-AI complementarity. For these benefits to materialize, explanations should enable humans to appropriately rely on AI recommendations and override the algorithmic recommendation when necessary to increase distributive fairness of decisions. The literature, however, does not provide conclusive empirical evidence as to whether explanations enable such complementarity in practice. In this work, we (a) provide a conceptual framework to articulate the relationships between explanations, fairness perceptions, reliance, and distributive fairness, (b) apply it to understand (seemingly) contradictory research findings at the intersection of explanations and fairness, and (c) derive cohesive implications for the formulation of research questions and the design of experiments.