B2Opt: Learning to Optimize Black-box Optimization with Little Budget
This addresses the problem of efficient optimization in resource-constrained settings for researchers and practitioners in fields like engineering and AI, though it appears incremental as it builds on genetic algorithm mechanisms.
The paper tackles the challenge of high-dimensional, expensive black-box optimization by proposing B2Opt, a deep neural network framework that learns optimization strategies from target or surrogate tasks, achieving multiple orders of magnitude performance improvement with less function evaluation cost compared to state-of-the-art baselines.
The core challenge of high-dimensional and expensive black-box optimization (BBO) is how to obtain better performance faster with little function evaluation cost. The essence of the problem is how to design an efficient optimization strategy tailored to the target task. This paper designs a powerful optimization framework to automatically learn the optimization strategies from the target or cheap surrogate task without human intervention. However, current methods are weak for this due to poor representation of optimization strategy. To achieve this, 1) drawing on the mechanism of genetic algorithm, we propose a deep neural network framework called B2Opt, which has a stronger representation of optimization strategies based on survival of the fittest; 2) B2Opt can utilize the cheap surrogate functions of the target task to guide the design of the efficient optimization strategies. Compared to the state-of-the-art BBO baselines, B2Opt can achieve multiple orders of magnitude performance improvement with less function evaluation cost. We validate our proposal on high-dimensional synthetic functions and two real-world applications. We also find that deep B2Opt performs better than shallow ones.