Actor Critic with Differentially Private Critic
This work addresses privacy concerns in reinforcement learning for domains like healthcare, offering an incremental improvement by integrating differential privacy into actor-critic methods.
The paper tackles the problem of sample inefficiency in reinforcement learning by enabling knowledge transfer from upstream tasks while protecting sensitive trajectory data, achieving improved sample efficiency in downstream control tasks with privacy guarantees.
Reinforcement learning algorithms are known to be sample inefficient, and often performance on one task can be substantially improved by leveraging information (e.g., via pre-training) on other related tasks. In this work, we propose a technique to achieve such knowledge transfer in cases where agent trajectories contain sensitive or private information, such as in the healthcare domain. Our approach leverages a differentially private policy evaluation algorithm to initialize an actor-critic model and improve the effectiveness of learning in downstream tasks. We empirically show this technique increases sample efficiency in resource-constrained control problems while preserving the privacy of trajectories collected in an upstream task.