Advancing Bayesian Optimization via Learning Correlated Latent Space
This addresses the optimization efficiency problem for researchers and practitioners working with structured or discrete data, though it appears incremental as it builds on existing latent space Bayesian optimization methods.
The paper tackles the problem of suboptimal solutions in Bayesian optimization when using latent spaces, by proposing CoBO which learns correlated latent spaces to minimize the inherent gap between latent and objective function distances. The method achieves high performance on tasks like molecule design and arithmetic expression fitting within a small budget.
Bayesian optimization is a powerful method for optimizing black-box functions with limited function evaluations. Recent works have shown that optimization in a latent space through deep generative models such as variational autoencoders leads to effective and efficient Bayesian optimization for structured or discrete data. However, as the optimization does not take place in the input space, it leads to an inherent gap that results in potentially suboptimal solutions. To alleviate the discrepancy, we propose Correlated latent space Bayesian Optimization (CoBO), which focuses on learning correlated latent spaces characterized by a strong correlation between the distances in the latent space and the distances within the objective function. Specifically, our method introduces Lipschitz regularization, loss weighting, and trust region recoordination to minimize the inherent gap around the promising areas. We demonstrate the effectiveness of our approach on several optimization tasks in discrete data, such as molecule design and arithmetic expression fitting, and achieve high performance within a small budget.