High-Dimensional Bayesian Optimisation with Variational Autoencoders and Deep Metric Learning
This addresses the problem of efficient optimisation in high-dimensional, data-scarce settings for researchers and practitioners in fields like drug discovery.
The paper tackles Bayesian optimisation in high-dimensional spaces by combining variational autoencoders and deep metric learning, achieving state-of-the-art results on a molecule generation benchmark with only 3% of the labelled data needed by prior methods.
We introduce a method combining variational autoencoders (VAEs) and deep metric learning to perform Bayesian optimisation (BO) over high-dimensional and structured input spaces. By adapting ideas from deep metric learning, we use label guidance from the blackbox function to structure the VAE latent space, facilitating the Gaussian process fit and yielding improved BO performance. Importantly for BO problem settings, our method operates in semi-supervised regimes where only few labelled data points are available. We run experiments on three real-world tasks, achieving state-of-the-art results on the penalised logP molecule generation benchmark using just 3% of the labelled data required by previous approaches. As a theoretical contribution, we present a proof of vanishing regret for VAE BO.