Federated Deep Reinforcement Learning
This addresses privacy-aware applications in deep reinforcement learning where data/model sharing is restricted, though it appears incremental as it adapts federated learning concepts to reinforcement learning.
The paper tackles the challenge of building high-quality policies in deep reinforcement learning when agents have limited training data and cannot share data/models due to privacy concerns, by proposing a Federated Deep Reinforcement Learning (FedRL) framework that uses Gaussian differentials to protect privacy and achieves competitive performance in Grid-world and Text2Action domains.
In deep reinforcement learning, building policies of high-quality is challenging when the feature space of states is small and the training data is limited. Despite the success of previous transfer learning approaches in deep reinforcement learning, directly transferring data or models from an agent to another agent is often not allowed due to the privacy of data and/or models in many privacy-aware applications. In this paper, we propose a novel deep reinforcement learning framework to federatively build models of high-quality for agents with consideration of their privacies, namely Federated deep Reinforcement Learning (FedRL). To protect the privacy of data and models, we exploit Gausian differentials on the information shared with each other when updating their local models. In the experiment, we evaluate our FedRL framework in two diverse domains, Grid-world and Text2Action domains, by comparing to various baselines.