Debiased Nonparametric Regression for Statistical Inference and Distributionally Robustness
This addresses a problem for researchers and practitioners in statistics and machine learning by enabling reliable statistical inference and robustness to covariate shift in nonparametric regression, though it is incremental as it builds on classical methods like Nadaraya-Watson regression.
The study tackles the lack of theoretical guarantees like pointwise/uniform risk convergence and asymptotic normality in modern nonparametric regression estimators (e.g., random forests, neural networks) by proposing a debiasing method that incorporates a correction term for the estimation error, resulting in a debiased estimator that satisfies these properties under mild smoothness conditions.
This study proposes a debiasing method for smooth nonparametric estimators. While machine learning techniques such as random forests and neural networks have demonstrated strong predictive performance, their theoretical properties remain relatively underexplored. In particular, many modern algorithms lack guarantees of pointwise and uniform risk convergence, as well as asymptotic normality. These properties are essential for statistical inference and robust estimation and have been well-established for classical methods such as Nadaraya-Watson regression. To ensure these properties for various nonparametric regression estimators, we introduce a model-free debiasing method. By incorporating a correction term that estimates the conditional expected residual of the original estimator, or equivalently, its estimation error, into the initial nonparametric regression estimator, we obtain a debiased estimator that satisfies pointwise and uniform risk convergence, along with asymptotic normality, under mild smoothness conditions. These properties facilitate statistical inference and enhance robustness to covariate shift, making the method broadly applicable to a wide range of nonparametric regression problems.