Sample-efficient reinforcement learning using deep Gaussian processes
This addresses the problem of costly interactions in reinforcement learning for applications like robotics, though it appears incremental as it builds on existing Gaussian process models.
The paper tackles the challenge of sample-efficient reinforcement learning in continuous control tasks by introducing deep Gaussian processes that incorporate prior knowledge on dynamics, resulting in highly improved early sample-efficiency over competing methods, as demonstrated across tasks including the half-cheetah.
Reinforcement learning provides a framework for learning to control which actions to take towards completing a task through trial-and-error. In many applications observing interactions is costly, necessitating sample-efficient learning. In model-based reinforcement learning efficiency is improved by learning to simulate the world dynamics. The challenge is that model inaccuracies rapidly accumulate over planned trajectories. We introduce deep Gaussian processes where the depth of the compositions introduces model complexity while incorporating prior knowledge on the dynamics brings smoothness and structure. Our approach is able to sample a Bayesian posterior over trajectories. We demonstrate highly improved early sample-efficiency over competing methods. This is shown across a number of continuous control tasks, including the half-cheetah whose contact dynamics have previously posed an insurmountable problem for earlier sample-efficient Gaussian process based models.