Kernel Autocovariance Operators of Stationary Processes: Estimation and Convergence
This work addresses theoretical foundations for kernel methods with dependent data, which is important for researchers in machine learning and statistics working with time series or sequential data.
The paper tackles the problem of estimating autocovariance operators for stationary stochastic processes in reproducing kernel Hilbert spaces, obtaining asymptotic convergence results and finite sample error bounds for ergodic and strongly mixing processes. The results provide consistency guarantees for kernel PCA with dependent data, conditional mean embeddings, and nonparametric estimation of Markov transition operators.
We consider autocovariance operators of a stationary stochastic process on a Polish space that is embedded into a reproducing kernel Hilbert space. We investigate how empirical estimates of these operators converge along realizations of the process under various conditions. In particular, we examine ergodic and strongly mixing processes and obtain several asymptotic results as well as finite sample error bounds. We provide applications of our theory in terms of consistency results for kernel PCA with dependent data and the conditional mean embedding of transition probabilities. Finally, we use our approach to examine the nonparametric estimation of Markov transition operators and highlight how our theory can give a consistency analysis for a large family of spectral analysis methods including kernel-based dynamic mode decomposition.