BREATHE: Second-Order Gradients and Heteroscedastic Emulation based Design Space Exploration
This addresses the challenge of efficient design space exploration for researchers in scientific and physical experiments, offering a novel method that extends beyond traditional vector spaces to graphs, though it appears incremental in combining existing techniques like gradients and surrogate models.
The paper tackles the problem of sample-inefficient exploration in large and complex design spaces, including vector and graph-based ones, by proposing BREATHE, a constrained multi-objective optimization framework that leverages second-order gradients and heteroscedastic surrogate models, resulting in performance gains such as up to 64.9% higher than baselines in graph-based search and up to 21.9× higher hypervolume in multi-objective tasks.
Researchers constantly strive to explore larger and more complex search spaces in various scientific studies and physical experiments. However, such investigations often involve sophisticated simulators or time-consuming experiments that make exploring and observing new design samples challenging. Previous works that target such applications are typically sample-inefficient and restricted to vector search spaces. To address these limitations, this work proposes a constrained multi-objective optimization (MOO) framework, called BREATHE, that searches not only traditional vector-based design spaces but also graph-based design spaces to obtain best-performing graphs. It leverages second-order gradients and actively trains a heteroscedastic surrogate model for sample-efficient optimization. In a single-objective vector optimization application, it leads to 64.1% higher performance than the next-best baseline, random forest regression. In graph-based search, BREATHE outperforms the next-best baseline, i.e., a graphical version of Gaussian-process-based Bayesian optimization, with up to 64.9% higher performance. In a MOO task, it achieves up to 21.9$\times$ higher hypervolume than the state-of-the-art method, multi-objective Bayesian optimization (MOBOpt). BREATHE also outperforms the baseline methods on most standard MOO benchmark applications.