Bayesian Optimisation over Multiple Continuous and Categorical Inputs
This addresses optimization challenges in domains like hyperparameter tuning or material design where mixed input types are common, but it appears incremental as it builds on existing Bayesian optimization and bandit techniques.
The paper tackles the problem of efficiently optimizing black-box functions with both continuous and categorical inputs by proposing CoCaBO, a method combining multi-armed bandits and Bayesian optimization, and demonstrates that it outperforms existing approaches on synthetic and real-world tasks.
Efficient optimisation of black-box problems that comprise both continuous and categorical inputs is important, yet poses significant challenges. We propose a new approach, Continuous and Categorical Bayesian Optimisation (CoCaBO), which combines the strengths of multi-armed bandits and Bayesian optimisation to select values for both categorical and continuous inputs. We model this mixed-type space using a Gaussian Process kernel, designed to allow sharing of information across multiple categorical variables, each with multiple possible values; this allows CoCaBO to leverage all available data efficiently. We extend our method to the batch setting and propose an efficient selection procedure that dynamically balances exploration and exploitation whilst encouraging batch diversity. We demonstrate empirically that our method outperforms existing approaches on both synthetic and real-world optimisation tasks with continuous and categorical inputs.