Bayesian Optimization in Variational Latent Spaces with Dynamic Compression
This addresses the challenge of adapting autonomous robots to new tasks and environments with very limited trials, though it is incremental as it builds on Bayesian optimization with trajectory-based kernels.
The paper tackled the problem of data-efficient optimization for robotics with only 10-20 trials by proposing a sequential variational autoencoder to embed simulated trajectories into a latent space and dynamically compress the search space, resulting in ultra data-efficient optimization validated on hardware and simulation experiments.
Data-efficiency is crucial for autonomous robots to adapt to new tasks and environments. In this work we focus on robotics problems with a budget of only 10-20 trials. This is a very challenging setting even for data-efficient approaches like Bayesian optimization (BO), especially when optimizing higher-dimensional controllers. Simulated trajectories can be used to construct informed kernels for BO. However, previous work employed supervised ways of extracting low-dimensional features for these. We propose a model and architecture for a sequential variational autoencoder that embeds the space of simulated trajectories into a lower-dimensional space of latent paths in an unsupervised way. We further compress the search space for BO by reducing exploration in parts of the state space that are undesirable, without requiring explicit constraints on controller parameters. We validate our approach with hardware experiments on a Daisy hexapod robot and an ABB Yumi manipulator. We also present simulation experiments with further comparisons to several baselines on Daisy and two manipulators. Our experiments indicate the proposed trajectory-based kernel with dynamic compression can offer ultra data-efficient optimization.