Regularized Modal Regression on Markov-dependent Observations: A Theoretical Assessment
This work addresses the theoretical underpinnings of robust regression for scenarios with dependent data, but it is incremental as it extends existing i.i.d. results to Markov-dependent settings.
The paper tackles the theoretical analysis of regularized modal regression under Markov-dependent observations, establishing an upper bound for the estimator and an explicit learning rate, showing that Markov dependence affects generalization error by discounting sample size based on the spectral gap of the Markov chain.
Modal regression, a widely used regression protocol, has been extensively investigated in statistical and machine learning communities due to its robustness to outliers and heavy-tailed noises. Understanding modal regression's theoretical behavior can be fundamental in learning theory. Despite significant progress in characterizing its statistical property, the majority of the results are based on the assumption that samples are independent and identical distributed (i.i.d.), which is too restrictive for real-world applications. This paper concerns the statistical property of regularized modal regression (RMR) within an important dependence structure - Markov dependent. Specifically, we establish the upper bound for RMR estimator under moderate conditions and give an explicit learning rate. Our results show that the Markov dependence impacts on the generalization error in the way that sample size would be discounted by a multiplicative factor depending on the spectral gap of underlying Markov chain. This result shed a new light on characterizing the theoretical underpinning for robust regression.