Bayesian Optimisation for Constrained Problems
This addresses constrained optimisation problems in fields like machine learning, offering an incremental improvement over existing methods.
The paper tackled the problem of optimizing expensive black-box functions with constraints, such as in hyperparameter tuning, by proposing a novel variant of the Knowledge Gradient acquisition function for Bayesian optimisation, and demonstrated its superior performance compared to four state-of-the-art algorithms while proving theoretical convergence.
Many real-world optimisation problems such as hyperparameter tuning in machine learning or simulation-based optimisation can be formulated as expensive-to-evaluate black-box functions. A popular approach to tackle such problems is Bayesian optimisation (BO), which builds a response surface model based on the data collected so far, and uses the mean and uncertainty predicted by the model to decide what information to collect next. In this paper, we propose a novel variant of the well-known Knowledge Gradient acquisition function that allows it to handle constraints. We empirically compare the new algorithm with four other state-of-the-art constrained Bayesian optimisation algorithms and demonstrate its superior performance. We also prove theoretical convergence in the infinite budget limit.