Local Entropy Search over Descent Sequences for Bayesian Optimization
This addresses the challenge of practical optimization in complex spaces for researchers and practitioners, though it is an incremental improvement over existing Bayesian optimization methods.
The paper tackles the problem of efficiently searching large design spaces by proposing local entropy search (LES), a Bayesian optimization method that targets solutions reachable by iterative optimizers like gradient descent, and it achieves strong sample efficiency in experiments on synthetic and benchmark problems.
Searching large and complex design spaces for a global optimum can be infeasible and unnecessary. A practical alternative is to iteratively refine the neighborhood of an initial design using local optimization methods such as gradient descent. We propose local entropy search (LES), a Bayesian optimization paradigm that explicitly targets the solutions reachable by the descent sequences of iterative optimizers. The algorithm propagates the posterior belief over the objective through the optimizer, resulting in a probability distribution over descent sequences. It then selects the next evaluation by maximizing mutual information with that distribution, using a combination of analytic entropy calculations and Monte-Carlo sampling of descent sequences. Empirical results on high-complexity synthetic objectives and benchmark problems show that LES achieves strong sample efficiency compared to existing local and global Bayesian optimization methods.