Robust Design and Evaluation of Predictive Algorithms under Unobserved Confounding
This work addresses a critical issue in algorithmic decision-making for domains like finance, where selective labels can lead to biased predictions and unfair outcomes, though it is incremental by formalizing existing empirical strategies.
The paper tackles the problem of evaluating predictive algorithms when outcomes are only observed for selected units, which creates bias due to unobserved confounding. It proposes a framework that bounds outcome differences between selected and unselected units, and demonstrates in a credit risk dataset that varying confounding assumptions significantly impacts predictions and fairness evaluations across income groups.
Predictive algorithms inform consequential decisions in settings with selective labels: outcomes are observed only for units selected by past decision makers. This creates an identification problem under unobserved confounding -- when selected and unselected units differ in unobserved ways that affect outcomes. We propose a framework for robust design and evaluation of predictive algorithms that bounds how much outcomes may differ between selected and unselected units with the same observed characteristics. These bounds formalize common empirical strategies including proxy outcomes and instrumental variables. Our estimators work across bounding strategies and performance measures such as conditional likelihoods, mean square error, and true/false positive rates. Using administrative data from a large Australian financial institution, we show that varying confounding assumptions substantially affects credit risk predictions and fairness evaluations across income groups.