Neural Generative Models for Global Optimization with Gradients
This work addresses the problem of efficiently exploring complex objective landscapes in global optimization for researchers and practitioners, though it is incremental as it builds on Evolutionary Search with neural enhancements.
The paper tackles global optimization for differentiable functions by proposing an Evolutionary Search-inspired method that models search distributions with Generative Neural Networks, achieving practical superiority over classical Evolutionary Search and gradient-based solutions on multimodal benchmarks and accelerating Bayesian Optimization with Gaussian Processes.
The aim of global optimization is to find the global optimum of arbitrary classes of functions, possibly highly multimodal ones. In this paper we focus on the subproblem of global optimization for differentiable functions and we propose an Evolutionary Search-inspired solution where we model point search distributions via Generative Neural Networks. This approach enables us to model diverse and complex search distributions based on which we can efficiently explore complicated objective landscapes. In our experiments we show the practical superiority of our algorithm versus classical Evolutionary Search and gradient-based solutions on a benchmark set of multimodal functions, and demonstrate how it can be used to accelerate Bayesian Optimization with Gaussian Processes.