Statistical Decision Theory with Counterfactual Loss
This work addresses a foundational gap in decision theory for fields like policy-making and causal inference, offering a novel framework to improve decision evaluation, though it is incremental in extending existing theory.
The paper tackles the limitation of classical statistical decision theory in evaluating decisions by incorporating counterfactual losses to assess quality relative to alternatives, proving that under strong ignorability, counterfactual risk is identifiable if and only if the loss function is additive in potential outcomes, and showing additive losses yield different treatment recommendations for multi-option problems.
Many researchers have applied classical statistical decision theory to evaluate treatment choices and learn optimal policies. However, because this framework is based solely on realized outcomes under chosen decisions and ignores counterfactual outcomes, it cannot assess the quality of a decision relative to feasible alternatives. For example, in bail decisions, a judge must consider not only crime prevention but also the avoidance of unnecessary burdens on arrestees. To address this limitation, we generalize standard decision theory by incorporating counterfactual losses, allowing decisions to be evaluated using all potential outcomes. The central challenge in this counterfactual statistical decision framework is identification: since only one potential outcome is observed for each unit, the associated counterfactual risk is generally not identifiable. We prove that, under the assumption of strong ignorability, the counterfactual risk is identifiable if and only if the counterfactual loss function is additive in the potential outcomes. Moreover, we demonstrate that additive counterfactual losses can yield treatment recommendations, which differ from those based on standard loss functions when the decision problem involves more than two treatment options. One interpretation of this result is that additive counterfactual losses can capture the accuracy and difficulty of a decision, whereas standard losses account for accuracy alone. Finally, we formulate a symbolic linear inverse program that, given a counterfactual loss, determines whether its risk is identifiable, without requiring data.