Asymptotic Bounds for Smoothness Parameter Estimates in Gaussian Process Interpolation
This addresses theoretical guarantees for parameter estimation in Gaussian process models, which is important for practitioners in fields like computer experiments, but it is incremental as it builds on existing approximation theory.
The paper tackles the problem of estimating the smoothness parameter in Gaussian process interpolation with Matérn kernels, proving that maximum likelihood estimation cannot asymptotically undersmooth the true function and recovers it for certain self-similar functions, while cross-validation provides a weaker lower bound.
It is common to model a deterministic response function, such as the output of a computer experiment, as a Gaussian process with a Matérn covariance kernel. The smoothness parameter of a Matérn kernel determines many important properties of the model in the large data limit, including the rate of convergence of the conditional mean to the response function. We prove that the maximum likelihood estimate of the smoothness parameter cannot asymptotically undersmooth the truth when the data are obtained on a fixed bounded subset of $\mathbb{R}^d$. That is, if the data-generating response function has Sobolev smoothness $ν_0 > d/2$, then the smoothness parameter estimate cannot be asymptotically less than $ν_0$. The lower bound is sharp. Additionally, we show that maximum likelihood estimation recovers the true smoothness for a class of compactly supported self-similar functions. For cross-validation we prove an asymptotic lower bound $ν_0 - d/2$, which however is unlikely to be sharp. The results are based on approximation theory in Sobolev spaces and some general theorems that restrict the set of values that the parameter estimators can take.