Stationarity without mean reversion in improper Gaussian processes
This addresses a foundational limitation in GP regression for machine learning applications where non-mean-reverting data is common, offering a novel theoretical extension.
The paper tackles the problem of stationary Gaussian processes (GPs) being mean-reverting, which can cause issues with data that does not relax to a fixed global mean, by introducing improper GP priors with infinite variance to define processes that are stationary but not mean-reverting. The result is a family of smooth non-reverting covariance functions that solve known pathologies while retaining favorable properties, as demonstrated on synthetic and real data.
The behavior of a GP regression depends on the choice of covariance function. Stationary covariance functions are preferred in machine learning applications. However, (non-periodic) stationary covariance functions are always mean reverting and can therefore exhibit pathological behavior when applied to data that does not relax to a fixed global mean value. In this paper we show that it is possible to use improper GP priors with infinite variance to define processes that are stationary but not mean reverting. To this aim, we use of non-positive kernels that can only be defined in this limit regime. The resulting posterior distributions can be computed analytically and it involves a simple correction of the usual formulas. The main contribution of the paper is the introduction of a large family of smooth non-reverting covariance functions that closely resemble the kernels commonly used in the GP literature (e.g. squared exponential and Matérn class). By analyzing both synthetic and real data, we demonstrate that these non-positive kernels solve some known pathologies of mean reverting GP regression while retaining most of the favorable properties of ordinary smooth stationary kernels.