Inference for relative sparsity
This work addresses uncertainty estimation for explainable treatment policies in healthcare, which is crucial for medical applications, but it is incremental as it builds on prior relative sparsity methods by adding inference and multi-stage extensions.
The paper tackles the problem of estimating treatment policies in healthcare that are constrained to be similar to the standard of care, focusing on developing inference methods for the relative sparsity objective to characterize uncertainty, and extends it to multi-stage decisions. It proposes a framework combining weighted Trust Region Policy Optimization, an adaptive penalty, and sample-splitting for post-selection inference, validated through asymptotic analysis, simulations, and real data.
In healthcare, there is much interest in estimating policies, or mappings from covariates to treatment decisions. Recently, there is also interest in constraining these estimated policies to the standard of care, which generated the observed data. A relative sparsity penalty was proposed to derive policies that have sparse, explainable differences from the standard of care, facilitating justification of the new policy. However, the developers of this penalty only considered estimation, not inference. Here, we develop inference for the relative sparsity objective function, because characterizing uncertainty is crucial to applications in medicine. Further, in the relative sparsity work, the authors only considered the single-stage decision case; here, we consider the more general, multi-stage case. Inference is difficult, because the relative sparsity objective depends on the unpenalized value function, which is unstable and has infinite estimands in the binary action case. Further, one must deal with a non-differentiable penalty. To tackle these issues, we nest a weighted Trust Region Policy Optimization function within a relative sparsity objective, implement an adaptive relative sparsity penalty, and propose a sample-splitting framework for post-selection inference. We study the asymptotic behavior of our proposed approaches, perform extensive simulations, and analyze a real, electronic health record dataset.