DTR Bandit: Learning to Make Response-Adaptive Decisions With Low Regret
This addresses the need for personalized, adaptive treatment plans in chronic conditions like depression, offering an online learning approach with theoretical guarantees, though it is incremental as it builds on existing DTR and bandit methods.
The paper tackles the problem of developing optimal dynamic treatment regimes (DTRs) online, where interactions affect both cumulative reward and future learning, by proposing a novel algorithm that achieves rate-optimal regret with linear models, demonstrated in synthetic experiments and a real-world case study on major depressive disorder.
Dynamic treatment regimes (DTRs) are personalized, adaptive, multi-stage treatment plans that adapt treatment decisions both to an individual's initial features and to intermediate outcomes and features at each subsequent stage, which are affected by decisions in prior stages. Examples include personalized first- and second-line treatments of chronic conditions like diabetes, cancer, and depression, which adapt to patient response to first-line treatment, disease progression, and individual characteristics. While existing literature mostly focuses on estimating the optimal DTR from offline data such as from sequentially randomized trials, we study the problem of developing the optimal DTR in an online manner, where the interaction with each individual affect both our cumulative reward and our data collection for future learning. We term this the DTR bandit problem. We propose a novel algorithm that, by carefully balancing exploration and exploitation, is guaranteed to achieve rate-optimal regret when the transition and reward models are linear. We demonstrate our algorithm and its benefits both in synthetic experiments and in a case study of adaptive treatment of major depressive disorder using real-world data.