The Minecraft Kernel: Modelling correlated Gaussian Processes in the Fourier domain
This work addresses a fundamental problem in multi-output Gaussian process modeling for researchers and practitioners in machine learning and statistics, representing a foundational advancement rather than an incremental improvement.
The authors tackled the challenge of modeling cross-covariances for multiple-output Gaussian processes using spectral mixture kernels, identifying a critical limitation where cross-covariances are not reproducible across all correlations except in special cases. They solved this by replacing Gaussian components with block components of finite bandwidth, enabling arbitrary approximation of any stationary multi-output kernel.
In the univariate setting, using the kernel spectral representation is an appealing approach for generating stationary covariance functions. However, performing the same task for multiple-output Gaussian processes is substantially more challenging. We demonstrate that current approaches to modelling cross-covariances with a spectral mixture kernel possess a critical blind spot. For a given pair of processes, the cross-covariance is not reproducible across the full range of permitted correlations, aside from the special case where their spectral densities are of identical shape. We present a solution to this issue by replacing the conventional Gaussian components of a spectral mixture with block components of finite bandwidth (i.e. rectangular step functions). The proposed family of kernel represents the first multi-output generalisation of the spectral mixture kernel that can approximate any stationary multi-output kernel to arbitrary precision.