StepCountJITAI: simulation environment for RL with application to physical activity adaptive intervention
This work addresses the problem of developing effective RL policies for just-in-time adaptive interventions in behavioral health, specifically for physical activity, but it is incremental as it focuses on creating a simulation tool rather than a new method or breakthrough.
The paper tackles the challenge of limited data and irrelevant simulation environments for reinforcement learning (RL) in physical activity adaptive interventions by introducing StepCountJITAI, an RL environment designed to foster research on optimal RL methods for this domain.
The use of reinforcement learning (RL) to learn policies for just-in-time adaptive interventions (JITAIs) is of significant interest in many behavioral intervention domains including improving levels of physical activity. In a messaging-based physical activity JITAI, a mobile health app is typically used to send messages to a participant to encourage engagement in physical activity. In this setting, RL methods can be used to learn what intervention options to provide to a participant in different contexts. However, deploying RL methods in real physical activity adaptive interventions comes with challenges: the cost and time constraints of real intervention studies result in limited data to learn adaptive intervention policies. Further, commonly used RL simulation environments have dynamics that are of limited relevance to physical activity adaptive interventions and thus shed little light on what RL methods may be optimal for this challenging application domain. In this paper, we introduce StepCountJITAI, an RL environment designed to foster research on RL methods that address the significant challenges of policy learning for adaptive behavioral interventions.