Dynamic Measurement Scheduling for Adverse Event Forecasting using Deep RL
This addresses the problem of reducing costs and improving accuracy in patient health monitoring for clinical settings, representing a novel application but with incremental methodological contributions.
The paper tackles the problem of inefficient clinical test scheduling by using deep reinforcement learning to strategically time fewer measurements, achieving higher predictive gain or lower cost compared to heuristic-based scheduling in simulations and providing useful insights for mortality forecasting on the MIMIC3 dataset.
Current clinical practice to monitor patients' health follows either regular or heuristic-based lab test (e.g. blood test) scheduling. Such practice not only gives rise to redundant measurements accruing cost, but may even lead to unnecessary patient discomfort. From the computational perspective, heuristic-based test scheduling might lead to reduced accuracy of clinical forecasting models. Computationally learning an optimal clinical test scheduling and measurement collection, is likely to lead to both, better predictive models and patient outcome improvement. We address the scheduling problem using deep reinforcement learning (RL) to achieve high predictive gain and low measurement cost, by scheduling fewer, but strategically timed tests. We first show that in the simulation our policy outperforms heuristic-based measurement scheduling with higher predictive gain or lower cost measured by accumulated reward. We then learn a scheduling policy for mortality forecasting in the real-world clinical dataset (MIMIC3), our learned policy is able to provide useful clinical insights. To our knowledge, this is the first RL application on multi-measurement scheduling problem in the clinical setting.