Bayesian optimization under mixed constraints with a slack-variable augmented Lagrangian
This work addresses constrained optimization in machine learning, offering an incremental improvement for scenarios with mixed constraints.
The paper tackled the problem of Bayesian optimization with mixed constraints by introducing a slack-variable augmented Lagrangian formulation, which allows for efficient evaluation of expected improvement using library routines and handles both equality and inequality constraints, showing favorable performance compared to existing methods.
An augmented Lagrangian (AL) can convert a constrained optimization problem into a sequence of simpler (e.g., unconstrained) problems, which are then usually solved with local solvers. Recently, surrogate-based Bayesian optimization (BO) sub-solvers have been successfully deployed in the AL framework for a more global search in the presence of inequality constraints; however, a drawback was that expected improvement (EI) evaluations relied on Monte Carlo. Here we introduce an alternative slack variable AL, and show that in this formulation the EI may be evaluated with library routines. The slack variables furthermore facilitate equality as well as inequality constraints, and mixtures thereof. We show how our new slack "ALBO" compares favorably to the original. Its superiority over conventional alternatives is reinforced on several mixed constraint examples.