Generalization bounds for nonparametric regression with $β-$mixing samples
This work provides a framework for analyzing regression errors in dependent data scenarios, extending classical theories like Vapnik-Chervonenkis to β-mixing samples, which is incremental as it builds on existing independent case results.
The paper tackles the problem of deriving generalization bounds for nonparametric regression when training samples are dependent, specifically for β-mixing sequences like geometrically ergodic Markov samples, by extending uniform deviation inequalities from the independent case to include additional error terms based on β-mixing coefficients.
In this paper we present a series of results that permit to extend in a direct manner uniform deviation inequalities of the empirical process from the independent to the dependent case characterizing the additional error in terms of $β-$mixing coefficients associated to the training sample. We then apply these results to some previously obtained inequalities for independent samples associated to the deviation of the least-squared error in nonparametric regression to derive corresponding generalization bounds for regression schemes in which the training sample may not be independent. These results provide a framework to analyze the error associated to regression schemes whose training sample comes from a large class of $β-$mixing sequences, including geometrically ergodic Markov samples, using only the independent case. More generally, they permit a meaningful extension of the Vapnik-Chervonenkis and similar theories for independent training samples to this class of $β-$mixing samples.