Evolving Gaussian Process kernels from elementary mathematical expressions
This addresses the challenge of kernel design for practitioners using Gaussian Processes, offering a more automated and potentially better-performing approach, though it is incremental as it builds on existing kernel composition methods.
The paper tackles the problem of kernel selection in Gaussian Processes by proposing a Genetic-Programming algorithm that evolves kernels from elementary mathematical expressions, achieving state-of-the-art results in real-world time-series extrapolation problems with reduced kernel complexity.
Choosing the most adequate kernel is crucial in many Machine Learning applications. Gaussian Process is a state-of-the-art technique for regression and classification that heavily relies on a kernel function. However, in the Gaussian Process literature, kernels have usually been either ad hoc designed, selected from a predefined set, or searched for in a space of compositions of kernels which have been defined a priori. In this paper, we propose a Genetic-Programming algorithm that represents a kernel function as a tree of elementary mathematical expressions. By means of this representation, a wider set of kernels can be modeled, where potentially better solutions can be found, although new challenges also arise. The proposed algorithm is able to overcome these difficulties and find kernels that accurately model the characteristics of the data. This method has been tested in several real-world time-series extrapolation problems, improving the state-of-the-art results while reducing the complexity of the kernels.