Counterfactual Predictions under Runtime Confounding
This addresses a key challenge in decision support systems where runtime confounding limits model fairness and accuracy, though it is incremental as it builds on existing counterfactual prediction methods.
The paper tackles the problem of counterfactual prediction when some relevant factors cannot be used at runtime due to ethical or practical constraints, proposing a doubly-robust method that often outperforms competing approaches in experiments.
Algorithms are commonly used to predict outcomes under a particular decision or intervention, such as predicting whether an offender will succeed on parole if placed under minimal supervision. Generally, to learn such counterfactual prediction models from observational data on historical decisions and corresponding outcomes, one must measure all factors that jointly affect the outcomes and the decision taken. Motivated by decision support applications, we study the counterfactual prediction task in the setting where all relevant factors are captured in the historical data, but it is either undesirable or impermissible to use some such factors in the prediction model. We refer to this setting as runtime confounding. We propose a doubly-robust procedure for learning counterfactual prediction models in this setting. Our theoretical analysis and experimental results suggest that our method often outperforms competing approaches. We also present a validation procedure for evaluating the performance of counterfactual prediction methods.