Deep Gaussian Processes for Multi-fidelity Modeling
This addresses the need for reliable models in machine learning and science applications where data acquisition costs vary, representing an incremental advance over existing methods.
The paper tackled the problem of combining cheap but noisy observations with limited expensive data in multi-fidelity modeling, resulting in a novel deep Gaussian process approach that improves uncertainty quantification and propagation, enhancing decision-making pipelines.
Multi-fidelity methods are prominently used when cheaply-obtained, but possibly biased and noisy, observations must be effectively combined with limited or expensive true data in order to construct reliable models. This arises in both fundamental machine learning procedures such as Bayesian optimization, as well as more practical science and engineering applications. In this paper we develop a novel multi-fidelity model which treats layers of a deep Gaussian process as fidelity levels, and uses a variational inference scheme to propagate uncertainty across them. This allows for capturing nonlinear correlations between fidelities with lower risk of overfitting than existing methods exploiting compositional structure, which are conversely burdened by structural assumptions and constraints. We show that the proposed approach makes substantial improvements in quantifying and propagating uncertainty in multi-fidelity set-ups, which in turn improves their effectiveness in decision making pipelines.