Expected Improvement via Gradient Norms
This work addresses a known bottleneck in Bayesian Optimization for expensive black-box function optimization, offering an incremental improvement over existing methods.
The paper tackles the problem of Bayesian Optimization (BO) being overly exploitative and converging to suboptimal points by proposing Expected Improvement via Gradient Norms (EI-GN), which promotes sampling in high-performing and stationary regions, resulting in consistent improvements on standard benchmarks.
Bayesian Optimization (BO) is a principled approach for optimizing expensive black-box functions, with Expected Improvement (EI) being one of the most widely used acquisition functions. Despite its empirical success, EI is known to be overly exploitative and can converge to suboptimal stationary points. We propose Expected Improvement via Gradient Norms (EI-GN), a novel acquisition function that applies the improvement principle to a gradient-aware auxiliary objective, thereby promoting sampling in regions that are both high-performing and approaching first-order stationarity. EI-GN relies on gradient observations used to learn gradient-enhanced surrogate models that enable principled gradient inference from function evaluations. We derive a tractable closed-form expression for EI-GN that allows efficient optimization and show that the proposed acquisition is consistent with the improvement-based acquisition framework. Empirical evaluations on standard BO benchmarks demonstrate that EI-GN yields consistent improvements against standard baselines. We further demonstrate applicability of EI-GN to control policy learning problems.