Identification of Subgroups With Similar Benefits in Off-Policy Policy Evaluation
This work addresses the need for personalization in decision policies while maintaining confidence in predictions, which is crucial for applications like healthcare or marketing, though it is incremental as it builds on existing HTE estimation methods.
The paper tackles the problem of accurately estimating heterogeneous treatment effects (HTEs) in off-policy policy evaluation for sequential decision making, where limited data can compromise individual predictions. It proposes a method that identifies subgroups with confident estimates of policy differences, showing improved accuracy in experiments compared to other methods.
Off-policy policy evaluation methods for sequential decision making can be used to help identify if a proposed decision policy is better than a current baseline policy. However, a new decision policy may be better than a baseline policy for some individuals but not others. This has motivated a push towards personalization and accurate per-state estimates of heterogeneous treatment effects (HTEs). Given the limited data present in many important applications, individual predictions can come at a cost to accuracy and confidence in such predictions. We develop a method to balance the need for personalization with confident predictions by identifying subgroups where it is possible to confidently estimate the expected difference in a new decision policy relative to a baseline. We propose a novel loss function that accounts for uncertainty during the subgroup partitioning phase. In experiments, we show that our method can be used to form accurate predictions of HTEs where other methods struggle.