Federated reinforcement learning for robot motion planning with zero-shot generalization
This work addresses the problem of efficient and private policy learning for robots in new environments, though it appears incremental as it builds on existing federated and reinforcement learning concepts.
The paper tackles robot motion planning with zero-shot generalization by developing a federated reinforcement learning framework that enables collaborative learning without sharing raw data, achieving theoretical guarantees on convergence and optimality, as validated through Monte Carlo simulation.
This paper considers the problem of learning a control policy for robot motion planning with zero-shot generalization, i.e., no data collection and policy adaptation is needed when the learned policy is deployed in new environments. We develop a federated reinforcement learning framework that enables collaborative learning of multiple learners and a central server, i.e., the Cloud, without sharing their raw data. In each iteration, each learner uploads its local control policy and the corresponding estimated normalized arrival time to the Cloud, which then computes the global optimum among the learners and broadcasts the optimal policy to the learners. Each learner then selects between its local control policy and that from the Cloud for next iteration. The proposed framework leverages on the derived zero-shot generalization guarantees on arrival time and safety. Theoretical guarantees on almost-sure convergence, almost consensus, Pareto improvement and optimality gap are also provided. Monte Carlo simulation is conducted to evaluate the proposed framework.