Harmonizable mixture kernels with variational Fourier features
This work addresses kernel design for Gaussian processes, which is a foundational issue in machine learning, but it appears incremental as it builds on existing spectral and mixture approaches.
The authors tackled the problem of limited expressive power in Gaussian processes by proposing the harmonizable mixture kernel (HMK), a novel family of non-stationary kernels derived from mixture models on the generalized spectral representation, and variational Fourier features, a sparse inference framework, resulting in a robust kernel learning framework that interpolates between local patterns.
The expressive power of Gaussian processes depends heavily on the choice of kernel. In this work we propose the novel harmonizable mixture kernel (HMK), a family of expressive, interpretable, non-stationary kernels derived from mixture models on the generalized spectral representation. As a theoretically sound treatment of non-stationary kernels, HMK supports harmonizable covariances, a wide subset of kernels including all stationary and many non-stationary covariances. We also propose variational Fourier features, an inter-domain sparse GP inference framework that offers a representative set of 'inducing frequencies'. We show that harmonizable mixture kernels interpolate between local patterns, and that variational Fourier features offers a robust kernel learning framework for the new kernel family.