Bayesian Optimization of Combinatorial Structures
This addresses a critical bottleneck in machine learning and engineering for tasks requiring optimization over discrete spaces, representing a novel advancement rather than an incremental improvement.
The paper tackles the problem of optimizing expensive black-box functions over combinatorial structures, which is challenging due to combinatorial explosion and costly evaluations, and proposes a new algorithm that consistently outperforms existing methods in experiments.
The optimization of expensive-to-evaluate black-box functions over combinatorial structures is an ubiquitous task in machine learning, engineering and the natural sciences. The combinatorial explosion of the search space and costly evaluations pose challenges for current techniques in discrete optimization and machine learning, and critically require new algorithmic ideas. This article proposes, to the best of our knowledge, the first algorithm to overcome these challenges, based on an adaptive, scalable model that identifies useful combinatorial structure even when data is scarce. Our acquisition function pioneers the use of semidefinite programming to achieve efficiency and scalability. Experimental evaluations demonstrate that this algorithm consistently outperforms other methods from combinatorial and Bayesian optimization.