Counterfactual Risk Assessments, Evaluation, and Fairness
This work addresses fairness and evaluation issues in high-stakes decision-making systems like criminal justice and healthcare, offering a novel approach but with incremental methodological contributions.
The paper tackles the problem that algorithmic risk assessments are trained and evaluated on historical data, which reflects risk under past policies rather than under the decision options they inform, by defining counterfactual analogues of performance and fairness metrics better suited for decision-making contexts. It introduces a doubly robust estimation method, shows that standard and counterfactual fairness metrics rarely align, and demonstrates empirical improvements on synthetic and real-world child welfare data.
Algorithmic risk assessments are increasingly used to help humans make decisions in high-stakes settings, such as medicine, criminal justice and education. In each of these cases, the purpose of the risk assessment tool is to inform actions, such as medical treatments or release conditions, often with the aim of reducing the likelihood of an adverse event such as hospital readmission or recidivism. Problematically, most tools are trained and evaluated on historical data in which the outcomes observed depend on the historical decision-making policy. These tools thus reflect risk under the historical policy, rather than under the different decision options that the tool is intended to inform. Even when tools are constructed to predict risk under a specific decision, they are often improperly evaluated as predictors of the target outcome. Focusing on the evaluation task, in this paper we define counterfactual analogues of common predictive performance and algorithmic fairness metrics that we argue are better suited for the decision-making context. We introduce a new method for estimating the proposed metrics using doubly robust estimation. We provide theoretical results that show that only under strong conditions can fairness according to the standard metric and the counterfactual metric simultaneously hold. Consequently, fairness-promoting methods that target parity in a standard fairness metric may --- and as we show empirically, do --- induce greater imbalance in the counterfactual analogue. We provide empirical comparisons on both synthetic data and a real world child welfare dataset to demonstrate how the proposed method improves upon standard practice.