Bayesian Optimization using Deep Gaussian Processes
This work addresses optimization challenges in domains like aerospace design where functions are non-stationary, though it appears incremental as it builds on existing Bayesian Optimization and Deep Gaussian Process techniques.
The paper tackles the problem of optimizing expensive black-box functions that are non-stationary, which classic Gaussian Processes struggle with due to stationarity assumptions, by proposing a Bayesian Optimization approach using Deep Gaussian Processes as surrogate models, and demonstrates its performance on test cases and an aerospace design problem compared to state-of-the-art methods.
Bayesian Optimization using Gaussian Processes is a popular approach to deal with the optimization of expensive black-box functions. However, because of the a priori on the stationarity of the covariance matrix of classic Gaussian Processes, this method may not be adapted for non-stationary functions involved in the optimization problem. To overcome this issue, a new Bayesian Optimization approach is proposed. It is based on Deep Gaussian Processes as surrogate models instead of classic Gaussian Processes. This modeling technique increases the power of representation to capture the non-stationarity by simply considering a functional composition of stationary Gaussian Processes, providing a multiple layer structure. This paper proposes a new algorithm for Global Optimization by coupling Deep Gaussian Processes and Bayesian Optimization. The specificities of this optimization method are discussed and highlighted with academic test cases. The performance of the proposed algorithm is assessed on analytical test cases and an aerospace design optimization problem and compared to the state-of-the-art stationary and non-stationary Bayesian Optimization approaches.