Personalized HeartSteps: A Reinforcement Learning Algorithm for Optimizing Physical Activity
This work addresses the problem of personalizing health interventions for users of mobile health apps, though it is incremental as it builds on prior data from HeartSteps V1.
The authors developed a reinforcement learning algorithm to optimize just-in-time adaptive interventions for promoting physical activity, which is being used in the HeartSteps V2 mobile app to deliver context-tailored activity suggestions five times daily.
With the recent evolution of mobile health technologies, health scientists are increasingly interested in developing just-in-time adaptive interventions (JITAIs), typically delivered via notification on mobile device and designed to help the user prevent negative health outcomes and promote the adoption and maintenance of healthy behaviors. A JITAI involves a sequence of decision rules (i.e., treatment policy) that takes the user's current context as input and specifies whether and what type of an intervention should be provided at the moment. In this paper, we develop a Reinforcement Learning (RL) algorithm that continuously learns and improves the treatment policy embedded in the JITAI as the data is being collected from the user. This work is motivated by our collaboration on designing the RL algorithm in HeartSteps V2 based on data from HeartSteps V1. HeartSteps is a physical activity mobile health application. The RL algorithm developed in this paper is being used in HeartSteps V2 to decide, five times per day, whether to deliver a context-tailored activity suggestion.