Kernel-Based Distributed Q-Learning: A Scalable Reinforcement Learning Approach for Dynamic Treatment Regimes
This work addresses the challenge of handling large datasets in medical applications for chronic disease treatment, though it appears incremental as it builds on existing Q-learning methods with a distributed and kernel-based approach.
The paper tackles the problem of scaling reinforcement learning for dynamic treatment regimes with large electronic health records by proposing a kernel-based distributed Q-learning algorithm, which reduces computational complexity compared to state-of-the-art deep reinforcement learning methods while maintaining comparable generalization performance in terms of accumulated rewards like survival time.
In recent years, large amounts of electronic health records (EHRs) concerning chronic diseases have been collected to facilitate medical diagnosis. Modeling the dynamic properties of EHRs related to chronic diseases can be efficiently done using dynamic treatment regimes (DTRs). While reinforcement learning (RL) is a widely used method for creating DTRs, there is ongoing research in developing RL algorithms that can effectively handle large amounts of data. In this paper, we present a scalable kernel-based distributed Q-learning algorithm for generating DTRs. We perform both theoretical assessments and numerical analysis for the proposed approach. The results demonstrate that our algorithm significantly reduces the computational complexity associated with the state-of-the-art deep reinforcement learning methods, while maintaining comparable generalization performance in terms of accumulated rewards across stages, such as survival time or cumulative survival probability.