Offline Reinforcement Learning with Differential Privacy
It addresses privacy concerns for sensitive applications like finance and healthcare, but is incremental as it builds on existing offline RL methods.
The paper tackles the problem of privacy risks in offline reinforcement learning by designing algorithms with differential privacy guarantees, achieving strong instance-dependent learning bounds and showing that privacy comes at almost no utility drop for medium-size datasets.
The offline reinforcement learning (RL) problem is often motivated by the need to learn data-driven decision policies in financial, legal and healthcare applications. However, the learned policy could retain sensitive information of individuals in the training data (e.g., treatment and outcome of patients), thus susceptible to various privacy risks. We design offline RL algorithms with differential privacy guarantees which provably prevent such risks. These algorithms also enjoy strong instance-dependent learning bounds under both tabular and linear Markov decision process (MDP) settings. Our theory and simulation suggest that the privacy guarantee comes at (almost) no drop in utility comparing to the non-private counterpart for a medium-size dataset.