Optimal Policies for the Homogeneous Selective Labels Problem
This addresses a fundamental challenge in consequential decision-making applications where only partial outcome data is available, though the analysis is limited to a simplified homogeneous setting without individual features.
This paper tackles the problem of learning decision policies when outcomes are only observed for selected decisions (selective labels), showing that for discounted total reward maximization the optimal policy is a threshold policy (optimal stopping problem), while for undiscounted infinite-horizon average reward optimal policies require positive acceptance probability in all states.
Selective labels are a common feature of consequential decision-making applications, referring to the lack of observed outcomes under one of the possible decisions. This paper reports work in progress on learning decision policies in the face of selective labels. The setting considered is both a simplified homogeneous one, disregarding individuals' features to facilitate determination of optimal policies, and an online one, to balance costs incurred in learning with future utility. For maximizing discounted total reward, the optimal policy is shown to be a threshold policy, and the problem is one of optimal stopping. In contrast, for undiscounted infinite-horizon average reward, optimal policies have positive acceptance probability in all states. Future work stemming from these results is discussed.