Online Regularized Statistical Learning in Reproducing Kernel Hilbert Space With Non-Stationary Data
This work addresses the challenge of statistical learning in dynamic environments for applications like streaming data analysis, but it appears incremental as it builds on existing regularization and RKHS methods.
The paper tackles the problem of online regularized learning in reproducing kernel Hilbert spaces with non-stationary data streams, showing that the tracking error vanishes in mean square under slowly time-varying conditions and achieving mean square consistency between the algorithm's output and the unknown function for independent and non-identically distributed data.
We study recursive regularized learning algorithms in the reproducing kernel Hilbert space (RKHS) with non-stationary online data streams. We introduce the concept of random Tikhonov regularization path and decompose the tracking error of the algorithm's output for the regularization path into random difference equations in RKHS. We show that the tracking error vanishes in mean square if the regularization path is slowly time-varying. Then, leveraging the monotonicity of inverse operators and the spectral decomposition of compact operators, and introducing the RKHS persistence of excitation condition, we develop a dominated convergence method to prove the mean square consistency between the regularization path and the unknown function to be learned. Especially, for independent and non-identically distributed data streams, the mean square consistency between the algorithm's output and the unknown function is achieved if the input data's marginal probability measures are slowly time-varying and the average measure over each fixed-length time period has a uniformly strictly positive lower bound.