Survivable Robotic Control through Guided Bayesian Policy Search with Deep Reinforcement Learning
This addresses the challenge of robotic survivability in dynamic, failure-prone environments, offering a domain-specific incremental improvement over existing reinforcement learning techniques.
The paper tackles the problem of enabling robots to sustain manipulation capabilities after mechanical failures by proposing Survivable Robotic Learning (SRL), which uses guided Bayesian policy search to adapt control policies with compromised degrees of freedom, resulting in improved sample efficiency and success ratios compared to baseline methods.
Many robot manipulation skills can be represented with deterministic characteristics and there exist efficient techniques for learning parameterized motor plans for those skills. However, one of the active research challenge still remains to sustain manipulation capabilities in situation of a mechanical failure. Ideally, like biological creatures, a robotic agent should be able to reconfigure its control policy by adapting to dynamic adversaries. In this paper, we propose a method that allows an agent to survive in a situation of mechanical loss, and adaptively learn manipulation with compromised degrees of freedom -- we call our method Survivable Robotic Learning (SRL). Our key idea is to leverage Bayesian policy gradient by encoding knowledge bias in posterior estimation, which in turn alleviates future policy search explorations, in terms of sample efficiency and when compared to random exploration based policy search methods. SRL represents policy priors as Gaussian process, which allows tractable computation of approximate posterior (when true gradient is intractable), by incorporating guided bias as proxy from prior replays. We evaluate our proposed method against off-the-shelf model free learning algorithm (DDPG), testing on a hexapod robot platform which encounters incremental failure emulation, and our experiments show that our method improves largely in terms of sample requirement and quantitative success ratio in all failure modes. A demonstration video of our experiments can be viewed at: https://sites.google.com/view/survivalrl