Efficient and Scalable Batch Bayesian Optimization Using K-Means
This work addresses the challenge of efficient batch optimization for real-world applications like drug discovery, though it appears incremental as it builds on existing BO methods with clustering and dimensionality reduction.
The authors tackled the problem of batch sampling in Bayesian Optimization by introducing KMBBO, which uses k-means clustering to estimate acquisition function peaks, and showed it outperforms state-of-the-art methods on tasks like hyper-parameter tuning and drug discovery. They also extended it with compressed sensing for high-dimensional data, demonstrating improved performance over non-reduction algorithms.
We present K-Means Batch Bayesian Optimization (KMBBO), a novel batch sampling algorithm for Bayesian Optimization (BO). KMBBO uses unsupervised learning to efficiently estimate peaks of the model acquisition function. We show in empirical experiments that our method outperforms the current state-of-the-art batch allocation algorithms on a variety of test problems including tuning of algorithm hyper-parameters and a challenging drug discovery problem. In order to accommodate the real-world problem of high dimensional data, we propose a modification to KMBBO by combining it with compressed sensing to project the optimization into a lower dimensional subspace. We demonstrate empirically that this 2-step method outperforms algorithms where no dimensionality reduction has taken place.