Bayesian Optimisation for Mixed-Variable Inputs using Value Proposals
This addresses a bottleneck in real-world optimization for domains like hyperparameter tuning, though it is incremental as it builds on prior bandit-based methods.
The paper tackles the problem of optimizing functions with both categorical and continuous variables by proposing a unified Bayesian optimization method that uses value proposals derived from the Expected Improvement criterion, showing it significantly outperforms existing approaches on mixed-variable black-box tasks.
Many real-world optimisation problems are defined over both categorical and continuous variables, yet efficient optimisation methods such asBayesian Optimisation (BO) are not designed tohandle such mixed-variable search spaces. Recent approaches to this problem cast the selection of the categorical variables as a bandit problem, operating independently alongside a BO component which optimises the continuous variables. In this paper, we adopt a holistic view and aim to consolidate optimisation of the categorical and continuous sub-spaces under a single acquisition metric. We derive candidates from the ExpectedImprovement criterion, which we call value proposals, and use these proposals to make selections on both the categorical and continuous components of the input. We show that this unified approach significantly outperforms existing mixed-variable optimisation approaches across several mixed-variable black-box optimisation tasks.