Regime-Adaptive Bayesian Optimization via Dirichlet Process Mixtures of Gaussian Processes
This addresses the challenge of optimizing complex, heterogeneous functions in fields like chemistry and engineering, though it is an incremental advancement building on existing BO and mixture model techniques.
The paper tackles the problem of Bayesian Optimization (BO) in multi-regime settings where uniform smoothness assumptions fail, such as in molecular conformation search and drug discovery, by proposing RAMBO, a Dirichlet Process Mixture of Gaussian Processes that automatically discovers latent regimes, resulting in consistent improvements over state-of-the-art baselines.
Standard Bayesian Optimization (BO) assumes uniform smoothness across the search space an assumption violated in multi-regime problems such as molecular conformation search through distinct energy basins or drug discovery across heterogeneous molecular scaffolds. A single GP either oversmooths sharp transitions or hallucinates noise in smooth regions, yielding miscalibrated uncertainty. We propose RAMBO, a Dirichlet Process Mixture of Gaussian Processes that automatically discovers latent regimes during optimization, each modeled by an independent GP with locally-optimized hyperparameters. We derive collapsed Gibbs sampling that analytically marginalizes latent functions for efficient inference, and introduce adaptive concentration parameter scheduling for coarse-to-fine regime discovery. Our acquisition functions decompose uncertainty into intra-regime and inter-regime components. Experiments on synthetic benchmarks and real-world applications, including molecular conformer optimization, virtual screening for drug discovery, and fusion reactor design, demonstrate consistent improvements over state-of-the-art baselines on multi-regime objectives.