Hybrid Parameter Search and Dynamic Model Selection for Mixed-Variable Bayesian Optimization
This work addresses the challenge of optimizing mixed-variable functions, which is incremental in improving surrogate models for Bayesian optimization.
The paper tackled the problem of Bayesian optimization with mixed variables by proposing a hybrid model that combines Monte Carlo Tree Search for categorical variables and Gaussian Processes for continuous ones, achieving superior performance in numerical experiments.
This paper presents a new type of hybrid model for Bayesian optimization (BO) adept at managing mixed variables, encompassing both quantitative (continuous and integer) and qualitative (categorical) types. Our proposed new hybrid models (named hybridM) merge the Monte Carlo Tree Search structure (MCTS) for categorical variables with Gaussian Processes (GP) for continuous ones. hybridM leverages the upper confidence bound tree search (UCTS) for MCTS strategy, showcasing the tree architecture's integration into Bayesian optimization. Our innovations, including dynamic online kernel selection in the surrogate modeling phase and a unique UCTS search strategy, position our hybrid models as an advancement in mixed-variable surrogate models. Numerical experiments underscore the superiority of hybrid models, highlighting their potential in Bayesian optimization.