Realistic CDSS Drug Dosing with End-to-end Recurrent Q-learning for Dual Vasopressor Control
This work addresses skepticism in clinical adoption of RL for drug dosing in intensive care, though it appears incremental by focusing on action space design and recurrent modeling.
The paper tackled the challenge of realistic drug dosing in clinical decision support systems by developing an end-to-end reinforcement learning approach for dual vasopressor control in ICU patients with septic shock, achieving over 15% improvement in survival probability while aligning with clinical protocols.
Reinforcement learning (RL) applications in Clinical Decision Support Systems (CDSS) frequently encounter skepticism from practitioners regarding inoperable dosing decisions. We address this challenge with an end-to-end approach for learning optimal drug dosing and control policies for dual vasopressor administration in intensive care unit (ICU) patients with septic shock. For realistic drug dosing, we apply action space design that accommodates discrete, continuous, and directional dosing strategies in a system that combines offline conservative Q-learning with a novel recurrent modeling in a replay buffer to capture temporal dependencies in ICU time-series data. Our comparative analysis of norepinephrine dosing strategies across different action space formulations reveals that the designed action spaces improve interpretability and facilitate clinical adoption while preserving efficacy. Empirical results1 on eICU and MIMIC demonstrate that action space design profoundly influences learned behavioral policies. The proposed methods achieve improved patient outcomes of over 15% in survival improvement probability, while aligning with established clinical protocols.