David Higdon

Christopher Beattie, David Higdon, Leanna House et al.

Gaussian processes (GPs) defined through intrinsic random fields provide a flexible framework for modeling spatial phenomena, and have been advocated in a variety of applications over the past several decades. Nevertheless, their adoption has lagged behind traditional, covariance-based approaches, in part because the intrinsic formulation has lacked an accompanying toolkit of computational methods and dependence specifications that facilitate fitting and prediction. We develop here a systematic framework for modeling intrinsic GPs and introduce practical algorithms and dependence/variogram models for modeling, inference and computation that parallel those of traditional, stationary GPs. We explore a close connection between intrinsic GP models and rational approximation, which clarifies the underlying problem structure. Numerical examples illustrate how the new tools can be deployed in practice, highlighting the advantages of intrinsic-field modeling in terms of robustness, interpretability, and computational efficiency.

Annie Sauer, Robert B. Gramacy, David Higdon

Deep Gaussian processes (DGPs) are increasingly popular as predictive models in machine learning (ML) for their non-stationary flexibility and ability to cope with abrupt regime changes in training data. Here we explore DGPs as surrogates for computer simulation experiments whose response surfaces exhibit similar characteristics. In particular, we transport a DGP's automatic warping of the input space and full uncertainty quantification (UQ), via a novel elliptical slice sampling (ESS) Bayesian posterior inferential scheme, through to active learning (AL) strategies that distribute runs non-uniformly in the input space -- something an ordinary (stationary) GP could not do. Building up the design sequentially in this way allows smaller training sets, limiting both expensive evaluation of the simulator code and mitigating cubic costs of DGP inference. When training data sizes are kept small through careful acquisition, and with parsimonious layout of latent layers, the framework can be both effective and computationally tractable. Our methods are illustrated on simulation data and two real computer experiments of varying input dimensionality. We provide an open source implementation in the "deepgp" package on CRAN.