Compressible Spectral Mixture Kernels with Sparse Dependency Structures for Gaussian Processes
This work addresses the need for more expressive and interpretable kernels in Gaussian processes for complex pattern modeling, representing an incremental improvement over existing spectral mixture kernels.
The paper tackles the challenge of improving Gaussian process generalization by introducing compressible spectral mixture kernels with sparse dependency structures, achieving enhanced performance on both synthetic and real-life applications.
Spectral mixture (SM) kernels comprise a powerful class of generalized kernels for Gaussian processes (GPs) to describe complex patterns. This paper introduces model compression and time- and phase (TP) modulated dependency structures to the original (SM) kernel for improved generalization of GPs. Specifically, by adopting Bienaymés identity, we generalize the dependency structure through cross-covariance between the SM components. Then, we propose a novel SM kernel with a dependency structure (SMD) by using cross-convolution between the SM components. Furthermore, we ameliorate the expressiveness of the dependency structure by parameterizing it with time and phase delays. The dependency structure has clear interpretations in terms of spectral density, covariance behavior, and sampling path. To enrich the SMD with effective hyperparameter initialization, compressible SM kernel components, and sparse dependency structures, we introduce a novel structure adaptation (SA) algorithm in the end. A thorough comparative analysis of the SMD on both synthetic and real-life applications corroborates its efficacy.