Safe global optimization of expensive noisy black-box functions in the $δ$-Lipschitz framework
This work addresses safe optimization for expensive noisy functions, which is crucial in applications like engineering design where failures are costly, but it is incremental as it builds on existing Lipschitz-based methods by adapting them to handle noise and safety constraints.
The paper tackles the problem of safe global maximization of expensive noisy black-box functions under Lipschitz conditions, where safety requires function evaluations to stay above a threshold, and proposes a δ-Lipschitz framework with two algorithms for determining safe regions and optimizing within them, supported by theoretical convergence results and numerical experiments.
In this paper, the problem of safe global maximization (it should not be confused with robust optimization) of expensive noisy black-box functions satisfying the Lipschitz condition is considered. The notion "safe" means that the objective function $f(x)$ during optimization should not violate a "safety" threshold, for instance, a certain a priori given value $h$ in a maximization problem. Thus, any new function evaluation (possibly corrupted by noise) must be performed at "safe points" only, namely, at points $y$ for which it is known that the objective function $f(y) > h$. The main difficulty here consists in the fact that the used optimization algorithm should ensure that the safety constraint will be satisfied at a point $y$ before evaluation of $f(y)$ will be executed. Thus, it is required both to determine the safe region $Ω$ within the search domain~$D$ and to find the global maximum within $Ω$. An additional difficulty consists in the fact that these problems should be solved in the presence of the noise. This paper starts with a theoretical study of the problem and it is shown that even though the objective function $f(x)$ satisfies the Lipschitz condition, traditional Lipschitz minorants and majorants cannot be used due to the presence of the noise. Then, a $δ$-Lipschitz framework and two algorithms using it are proposed to solve the safe global maximization problem. The first method determines the safe area within the search domain and the second one executes the global maximization over the found safe region. For both methods a number of theoretical results related to their functioning and convergence is established. Finally, numerical experiments confirming the reliability of the proposed procedures are performed.