Designing and evaluating an online reinforcement learning agent for physical exercise recommendations in N-of-1 trials
This work addresses the challenge of implementing and assessing personalized interventions in clinical settings, though it is incremental as it builds on existing reinforcement learning methods for N-of-1 trials.
The authors tackled the problem of evaluating personalized adaptive interventions for physical exercise recommendations in endometriosis pain management using an online reinforcement learning agent, finding that such interventions are feasible and can improve patient benefits even with limited data.
Personalized adaptive interventions offer the opportunity to increase patient benefits, however, there are challenges in their planning and implementation. Once implemented, it is an important question whether personalized adaptive interventions are indeed clinically more effective compared to a fixed gold standard intervention. In this paper, we present an innovative N-of-1 trial study design testing whether implementing a personalized intervention by an online reinforcement learning agent is feasible and effective. Throughout, we use a new study on physical exercise recommendations to reduce pain in endometriosis for illustration. We describe the design of a contextual bandit recommendation agent and evaluate the agent in simulation studies. The results show that, first, implementing a personalized intervention by an online reinforcement learning agent is feasible. Second, such adaptive interventions have the potential to improve patients' benefits even if only few observations are available. As one challenge, they add complexity to the design and implementation process. In order to quantify the expected benefit, data from previous interventional studies is required. We expect our approach to be transferable to other interventions and clinical interventions.