Learn to Intervene: An Adaptive Learning Policy for Restless Bandits in Application to Preventive Healthcare
This addresses the challenge of maintaining patient adherence in public health settings, such as automated call-based programs for pregnant women, but is incremental as it builds on existing RMAB and learning methods.
The paper tackles the problem of patient disengagement from preventive healthcare programs by modeling it as a restless multi-armed bandit (RMAB) with unknown transition probabilities, proposing a Whittle index based Q-Learning mechanism that converges to the optimal solution and improves over existing methods on benchmarks and a maternal healthcare dataset.
In many public health settings, it is important for patients to adhere to health programs, such as taking medications and periodic health checks. Unfortunately, beneficiaries may gradually disengage from such programs, which is detrimental to their health. A concrete example of gradual disengagement has been observed by an organization that carries out a free automated call-based program for spreading preventive care information among pregnant women. Many women stop picking up calls after being enrolled for a few months. To avoid such disengagements, it is important to provide timely interventions. Such interventions are often expensive and can be provided to only a small fraction of the beneficiaries. We model this scenario as a restless multi-armed bandit (RMAB) problem, where each beneficiary is assumed to transition from one state to another depending on the intervention. Moreover, since the transition probabilities are unknown a priori, we propose a Whittle index based Q-Learning mechanism and show that it converges to the optimal solution. Our method improves over existing learning-based methods for RMABs on multiple benchmarks from literature and also on the maternal healthcare dataset.