Transfer Learning Across Patient Variations with Hidden Parameter Markov Decision Processes
This work addresses personalized medicine for HIV patients, but it is incremental as it builds on an existing HiP-MDP framework.
The paper tackled the problem of patient variation in treatment responses by updating the Hidden Parameter Markov Decision Process (HiP-MDP) framework to model embedding uncertainty jointly with state uncertainty, enabling more robust personalized HIV treatment strategies.
Due to physiological variation, patients diagnosed with the same condition may exhibit divergent, but related, responses to the same treatments. Hidden Parameter Markov Decision Processes (HiP-MDPs) tackle this transfer-learning problem by embedding these tasks into a low-dimensional space. However, the original formulation of HiP-MDP had a critical flaw: the embedding uncertainty was modeled independently of the agent's state uncertainty, requiring an unnatural training procedure in which all tasks visited every part of the state space---possible for robots that can be moved to a particular location, impossible for human patients. We update the HiP-MDP framework and extend it to more robustly develop personalized medicine strategies for HIV treatment.