Neural Non-Stationary Spectral Kernel
This work addresses the need for more flexible kernels in Gaussian processes for machine learning practitioners, though it is incremental as it builds on existing spectral mixture and non-stationary kernel methods.
The paper tackled the problem of learning complex patterns in data by proposing a neural network-based non-stationary spectral mixture kernel for Gaussian processes, achieving the best performance on several benchmark datasets compared to stationary and other non-stationary variants.
Standard kernels such as Matérn or RBF kernels only encode simple monotonic dependencies within the input space. Spectral mixture kernels have been proposed as general-purpose, flexible kernels for learning and discovering more complicated patterns in the data. Spectral mixture kernels have recently been generalized into non-stationary kernels by replacing the mixture weights, frequency means and variances by input-dependent functions. These functions have also been modelled as Gaussian processes on their own. In this paper we propose modelling the hyperparameter functions with neural networks, and provide an experimental comparison between the stationary spectral mixture and the two non-stationary spectral mixtures. Scalable Gaussian process inference is implemented within the sparse variational framework for all the kernels considered. We show that the neural variant of the kernel is able to achieve the best performance, among alternatives, on several benchmark datasets.