On the Identifiability and Interpretability of Gaussian Process Models
This work addresses kernel selection problems for researchers and practitioners using Gaussian process models, but it is incremental as it builds on existing kernel theory with specific theoretical extensions.
The paper tackles the identifiability and interpretability issues in Gaussian process models by analyzing additive mixtures of Matérn kernels for single-output cases, showing that parameters are not identifiable and smoothness is determined by the least smooth component, and by proving that multiplicative mixtures for multi-output cases have identifiable covariance matrices up to a constant, supported by simulations and real applications.
In this paper, we critically examine the prevalent practice of using additive mixtures of Matérn kernels in single-output Gaussian process (GP) models and explore the properties of multiplicative mixtures of Matérn kernels for multi-output GP models. For the single-output case, we derive a series of theoretical results showing that the smoothness of a mixture of Matérn kernels is determined by the least smooth component and that a GP with such a kernel is effectively equivalent to the least smooth kernel component. Furthermore, we demonstrate that none of the mixing weights or parameters within individual kernel components are identifiable. We then turn our attention to multi-output GP models and analyze the identifiability of the covariance matrix $A$ in the multiplicative kernel $K(x,y) = AK_0(x,y)$, where $K_0$ is a standard single output kernel such as Matérn. We show that $A$ is identifiable up to a multiplicative constant, suggesting that multiplicative mixtures are well suited for multi-output tasks. Our findings are supported by extensive simulations and real applications for both single- and multi-output settings. This work provides insight into kernel selection and interpretation for GP models, emphasizing the importance of choosing appropriate kernel structures for different tasks.