Fast Model-based Policy Search for Universal Policy Networks
This work addresses a specific efficiency problem in physics-based reinforcement learning for adapting agents to new environments, representing an incremental improvement.
The paper tackles the challenge of efficiently selecting the most appropriate policy from a universal policy network for new environments, and shows that their Gaussian Process-based prior integrated with Bayesian Optimization outperforms other baselines in various control environments.
Adapting an agent's behaviour to new environments has been one of the primary focus areas of physics based reinforcement learning. Although recent approaches such as universal policy networks partially address this issue by enabling the storage of multiple policies trained in simulation on a wide range of dynamic/latent factors, efficiently identifying the most appropriate policy for a given environment remains a challenge. In this work, we propose a Gaussian Process-based prior learned in simulation, that captures the likely performance of a policy when transferred to a previously unseen environment. We integrate this prior with a Bayesian Optimisation-based policy search process to improve the efficiency of identifying the most appropriate policy from the universal policy network. We empirically evaluate our approach in a range of continuous and discrete control environments, and show that it outperforms other competing baselines.