RL4health: Crowdsourcing Reinforcement Learning for Knee Replacement Pathway Optimization
This work addresses cost reduction in knee replacement pathways, a common inpatient surgery, but is incremental as it applies existing reinforcement learning methods to a specific healthcare domain.
The paper tackled the problem of optimizing clinical pathways for knee replacement surgery by framing it as a sequential decision process and learning an optimal policy from historical claims data, resulting in a 7% reduction in overall cost and a 33% reduction in excessive cost premium.
Joint replacement is the most common inpatient surgical treatment in the US. We investigate the clinical pathway optimization for knee replacement, which is a sequential decision process from onset to recovery. Based on episodic claims from previous cases, we view the pathway optimization as an intelligence crowdsourcing problem and learn the optimal decision policy from data by imitating the best expert at every intermediate state. We develop a reinforcement learning-based pipeline that uses value iteration, state compression and aggregation learning, kernel representation and cross validation to predict the best treatment policy. It also provides forecast of the clinical pathway under the optimized policy. Empirical validation shows that the optimized policy reduces the overall cost by 7 percent and reduces the excessive cost premium by 33 percent.