Enhancing Adaptive Behavioral Interventions with LLM Inference from Participant-Described States
This work addresses data efficiency issues for health and behavioral science researchers developing personalized interventions, though it is incremental as it builds on existing RL and LLM methods.
The paper tackles the problem of data scarcity in reinforcement learning for adaptive health interventions by allowing participants to describe their states in natural language and using LLM inference to align policies with these descriptions, showing potential for significant performance improvements in a simulated physical activity environment.
The use of reinforcement learning (RL) methods to support health behavior change via personalized and just-in-time adaptive interventions is of significant interest to health and behavioral science researchers focused on problems such as smoking cessation support and physical activity promotion. However, RL methods are often applied to these domains using a small collection of context variables to mitigate the significant data scarcity issues that arise from practical limitations on the design of adaptive intervention trials. In this paper, we explore an approach to significantly expanding the state space of an adaptive intervention without impacting data efficiency. The proposed approach enables intervention participants to provide natural language descriptions of aspects of their current state. It then leverages inference with pre-trained large language models (LLMs) to better align the policy of a base RL method with these state descriptions. To evaluate our method, we develop a novel physical activity intervention simulation environment that generates text-based state descriptions conditioned on latent state variables using an auxiliary LLM. We show that this approach has the potential to significantly improve the performance of online policy learning methods.