Posterior Covariance Structures in Gaussian Processes
This work addresses covariance estimation in Gaussian processes, offering incremental improvements for applications like matrix approximation and preconditioning.
The paper analyzes the posterior covariance field in Gaussian processes, revealing how kernel parameters and observation distributions affect covariance structures, and proposes estimators for efficient approximation and preconditioning of covariance matrices.
In this paper, we present a comprehensive analysis of the posterior covariance field in Gaussian processes, with applications to the posterior covariance matrix. The analysis is based on the Gaussian prior covariance but the approach also applies to other covariance kernels. Our geometric analysis reveals how the Gaussian kernel's bandwidth parameter and the spatial distribution of the observations influence the posterior covariance as well as the corresponding covariance matrix, enabling straightforward identification of areas with high or low covariance in magnitude. Drawing inspiration from the a posteriori error estimation techniques in adaptive finite element methods, we also propose several estimators to efficiently measure the absolute posterior covariance field, which can be used for efficient covariance matrix approximation and preconditioning. We conduct a wide range of experiments to illustrate our theoretical findings and their practical applications.