Deep Kernels for Optimizing Locomotion Controllers
This work addresses the challenge of expensive hardware experiments for robotics researchers by providing a method to enhance optimization efficiency, though it is incremental as it builds on prior Bayesian Optimization approaches.
The paper tackles the problem of sample efficiency in optimizing locomotion controllers by automatically learning a distance metric using a neural network from simulation data, resulting in improved sample efficiency for both a 5-dimensional controller on ATRIAS hardware and a 16-dimensional controller in simulations.
Sample efficiency is important when optimizing parameters of locomotion controllers, since hardware experiments are time consuming and expensive. Bayesian Optimization, a sample-efficient optimization framework, has recently been widely applied to address this problem, but further improvements in sample efficiency are needed for practical applicability to real-world robots and high-dimensional controllers. To address this, prior work has proposed using domain expertise for constructing custom distance metrics for locomotion. In this work we show how to learn such a distance metric automatically. We use a neural network to learn an informed distance metric from data obtained in high-fidelity simulations. We conduct experiments on two different controllers and robot architectures. First, we demonstrate improvement in sample efficiency when optimizing a 5-dimensional controller on the ATRIAS robot hardware. We then conduct simulation experiments to optimize a 16-dimensional controller for a 7-link robot model and obtain significant improvements even when optimizing in perturbed environments. This demonstrates that our approach is able to enhance sample efficiency for two different controllers, hence is a fitting candidate for further experiments on hardware in the future.