Interaction-Limited Safe Continuous-Time RL for Dynamical Medical Treatment
For medical treatment decision-making, this work addresses the practical need for safe, adaptive interaction timing in continuous-time settings, offering a framework with theoretical guarantees.
The paper tackles safe continuous-time reinforcement learning for dynamic medical treatment, jointly optimizing treatment administration and clinical interaction timing under trajectory-level safety constraints. The proposed method improves safety and treatment effectiveness over equidistant interaction schemes across different safe policy optimization methods.
Dynamic medical treatment requires deciding treatment intensity and intervention timing, while patient states evolve continuously and adverse events may occur between clinical interactions. Most existing treatment learning methods assume fixed schedules or enforce safety only at discrete decision points. We propose Interaction-Limited Safe Continuous-Time Reinforcement Learning, a framework that jointly optimizes treatment administration and clinical interaction timing under trajectory-level safety constraints. Our key idea is to reformulate the continuous time treatment problem as an option-based semi-Markov decision process, where each option specifies a continuous-time treatment policy and its duration. We develop a safety-tightening mechanism showing that suitably constructed constraints at interaction times guarantee safety over the full continuous-time trajectory with high probability. We further establish finite-sample guarantees for policy learning from logged treatment trajectories and introduce a practical data-driven conservative surrogate. Experiments show that the proposed adaptive interaction-timing mechanism improves both safety and treatment effectiveness over equidistant interaction schemes across different safe policy optimization methods.