Active learning for enumerating local minima based on Gaussian process derivatives
This addresses a challenge in optimization for researchers or practitioners dealing with complex, unobservable derivative functions, but it appears incremental as it builds on existing active learning and Gaussian Process methods.
The paper tackles the problem of efficiently enumerating all local minima of a black-box function using active learning based on Gaussian Processes, where derivatives cannot be directly observed, and demonstrates its usefulness through numerical experiments.
We study active learning (AL) based on Gaussian Processes (GPs) for efficiently enumerating all of the local minimum solutions of a black-box function. This problem is challenging due to the fact that local solutions are characterized by their zero gradient and positive-definite Hessian properties, but those derivatives cannot be directly observed. We propose a new AL method in which the input points are sequentially selected such that the confidence intervals of the GP derivatives are effectively updated for enumerating local minimum solutions. We theoretically analyze the proposed method and demonstrate its usefulness through numerical experiments.