Causal Context Connects Counterfactual Fairness to Robust Prediction and Group Fairness
This work addresses the problem of making counterfactual fairness more practical for AI developers and policymakers by linking it to observable group fairness metrics, though it is incremental in bridging existing concepts.
The paper tackles the challenge of implementing counterfactual fairness in AI systems by introducing causal context to connect it with robust prediction and group fairness metrics, showing that under certain conditions, counterfactual fairness can be accuracy-optimal and equivalent to metrics like demographic parity, equalized odds, and calibration in common fairness contexts.
Counterfactual fairness requires that a person would have been classified in the same way by an AI or other algorithmic system if they had a different protected class, such as a different race or gender. This is an intuitive standard, as reflected in the U.S. legal system, but its use is limited because counterfactuals cannot be directly observed in real-world data. On the other hand, group fairness metrics (e.g., demographic parity or equalized odds) are less intuitive but more readily observed. In this paper, we use $\textit{causal context}$ to bridge the gaps between counterfactual fairness, robust prediction, and group fairness. First, we motivate counterfactual fairness by showing that there is not necessarily a fundamental trade-off between fairness and accuracy because, under plausible conditions, the counterfactually fair predictor is in fact accuracy-optimal in an unbiased target distribution. Second, we develop a correspondence between the causal graph of the data-generating process and which, if any, group fairness metrics are equivalent to counterfactual fairness. Third, we show that in three common fairness contexts$\unicode{x2013}$measurement error, selection on label, and selection on predictors$\unicode{x2013}$counterfactual fairness is equivalent to demographic parity, equalized odds, and calibration, respectively. Counterfactual fairness can sometimes be tested by measuring relatively simple group fairness metrics.