Exploiting Structure for Fast Kernel Learning
This work addresses scalability challenges in Gaussian process modeling for large-scale applications like climate modeling and video processing, representing a domain-specific advancement.
The authors tackled the problem of exact Gaussian process inference on massive datasets with missing values by proposing two novel methods that exploit Kronecker matrix algebra for scalability, achieving exact inference on up to 1 billion points in video reconstruction.
We propose two methods for exact Gaussian process (GP) inference and learning on massive image, video, spatial-temporal, or multi-output datasets with missing values (or "gaps") in the observed responses. The first method ignores the gaps using sparse selection matrices and a highly effective low-rank preconditioner is introduced to accelerate computations. The second method introduces a novel approach to GP training whereby response values are inferred on the gaps before explicitly training the model. We find this second approach to be greatly advantageous for the class of problems considered. Both of these novel approaches make extensive use of Kronecker matrix algebra to design massively scalable algorithms which have low memory requirements. We demonstrate exact GP inference for a spatial-temporal climate modelling problem with 3.7 million training points as well as a video reconstruction problem with 1 billion points.