Noise Estimation in Gaussian Process Regression
This work addresses a specific computational bottleneck in Gaussian process regression for practitioners in fields like statistics and machine learning, though it appears incremental.
The authors tackled the problem of estimating covariance hyperparameters in Gaussian process regression models with additive noise by developing a method that reduces the dimensionality to a univariate root-finding problem, demonstrating computational advantages and robustness compared to traditional optimization.
We develop a computational procedure to estimate the covariance hyperparameters for semiparametric Gaussian process regression models with additive noise. Namely, the presented method can be used to efficiently estimate the variance of the correlated error, and the variance of the noise based on maximizing a marginal likelihood function. Our method involves suitably reducing the dimensionality of the hyperparameter space to simplify the estimation procedure to a univariate root-finding problem. Moreover, we derive bounds and asymptotes of the marginal likelihood function and its derivatives, which are useful to narrowing the initial range of the hyperparameter search. Using numerical examples, we demonstrate the computational advantages and robustness of the presented approach compared to traditional parameter optimization.