Heteroscedastic Treed Bayesian Optimisation
This work addresses a key bottleneck in Bayesian optimization for practitioners in fields such as machine learning and mining, though it appears incremental as it builds on existing methods to handle heteroscedasticity.
The paper tackled the problem of non-stationarity in Bayesian optimization, which negatively affects performance, by proposing a novel prior model with hierarchical parameter learning, resulting in substantial improvements across applications like automatic machine learning and mining exploration.
Optimising black-box functions is important in many disciplines, such as tuning machine learning models, robotics, finance and mining exploration. Bayesian optimisation is a state-of-the-art technique for the global optimisation of black-box functions which are expensive to evaluate. At the core of this approach is a Gaussian process prior that captures our belief about the distribution over functions. However, in many cases a single Gaussian process is not flexible enough to capture non-stationarity in the objective function. Consequently, heteroscedasticity negatively affects performance of traditional Bayesian methods. In this paper, we propose a novel prior model with hierarchical parameter learning that tackles the problem of non-stationarity in Bayesian optimisation. Our results demonstrate substantial improvements in a wide range of applications, including automatic machine learning and mining exploration.