BayeSQP: Bayesian Optimization through Sequential Quadratic Programming
This work addresses optimization challenges for researchers and practitioners dealing with high-dimensional black-box functions, though it appears incremental as it combines existing techniques.
The paper tackles the problem of general black-box optimization by introducing BayeSQP, which merges sequential quadratic programming with Bayesian optimization, resulting in a method that outperforms state-of-the-art approaches in specific high-dimensional settings.
We introduce BayeSQP, a novel algorithm for general black-box optimization that merges the structure of sequential quadratic programming with concepts from Bayesian optimization. BayeSQP employs second-order Gaussian process surrogates for both the objective and constraints to jointly model the function values, gradients, and Hessian from only zero-order information. At each iteration, a local subproblem is constructed using the GP posterior estimates and solved to obtain a search direction. Crucially, the formulation of the subproblem explicitly incorporates uncertainty in both the function and derivative estimates, resulting in a tractable second-order cone program for high probability improvements under model uncertainty. A subsequent one-dimensional line search via constrained Thompson sampling selects the next evaluation point. Empirical results show thatBayeSQP outperforms state-of-the-art methods in specific high-dimensional settings. Our algorithm offers a principled and flexible framework that bridges classical optimization techniques with modern approaches to black-box optimization.