Sample-efficient Bayesian Optimisation Using Known Invariances
This work addresses sample efficiency in Bayesian optimization for researchers and practitioners dealing with invariant functions, offering incremental improvements through kernel modifications.
The paper tackles the inefficiency of Bayesian optimization (BO) for functions with known invariances by incorporating group invariances into Gaussian process kernels, achieving significant improvements in sample efficiency, as demonstrated on synthetic functions and a nuclear fusion reactor design where non-invariant methods failed.
Bayesian optimisation (BO) is a powerful framework for global optimisation of costly functions, using predictions from Gaussian process models (GPs). In this work, we apply BO to functions that exhibit invariance to a known group of transformations. We show that vanilla and constrained BO algorithms are inefficient when optimising such invariant objectives, and provide a method for incorporating group invariances into the kernel of the GP to produce invariance-aware algorithms that achieve significant improvements in sample efficiency. We derive a bound on the maximum information gain of these invariant kernels, and provide novel upper and lower bounds on the number of observations required for invariance-aware BO algorithms to achieve $ε$-optimality. We demonstrate our method's improved performance on a range of synthetic invariant and quasi-invariant functions. We also apply our method in the case where only some of the invariance is incorporated into the kernel, and find that these kernels achieve similar gains in sample efficiency at significantly reduced computational cost. Finally, we use invariant BO to design a current drive system for a nuclear fusion reactor, finding a high-performance solution where non-invariant methods failed.