On Safer Reinforcement Learning Policies for Sedation and Analgesia in Intensive Care
This work addresses safer pain management for ICU patients, though it is incremental by building on prior reinforcement learning methods with a focus on long-term outcomes.
The study tackled the problem of optimizing sedation and analgesia dosing in intensive care using reinforcement learning, finding that policies valuing both pain reduction and mortality led to safer outcomes, with actions negatively correlated with mortality compared to policies focusing only on pain.
Pain management in intensive care usually involves complex trade-offs between therapeutic goals and patient safety, since both inadequate and excessive treatment may induce serious sequelae. Reinforcement learning can help address this challenge by learning medication dosing policies from retrospective data. However, prior work on sedation and analgesia has optimized for objectives that do not value patient survival while relying on algorithms unsuitable for imperfect information settings. We investigated the risks of these design choices by implementing a deep reinforcement learning framework to suggest hourly medication doses under partial observability. Using data from 47,144 ICU stays in the MIMIC-IV database, we trained policies to prescribe opioids, propofol, benzodiazepines, and dexmedetomidine according to two goals: reduce pain or jointly reduce pain and mortality. We found that, although the two policies were associated with lower pain, actions from the first policy were positively correlated with mortality, while those proposed by the second policy were negatively correlated. This suggests that valuing long-term outcomes could be critical for safer treatment policies, even if a short-term goal remains the primary objective.