Robust Nonparametric Regression under Huber's $ε$-contamination Model
This addresses robust estimation in noisy data for statistical learning, offering incremental improvements by adapting existing methods to adversarial settings.
The paper tackles robust nonparametric regression under Huber's ε-contamination model, showing that a local binning median step effectively removes adversarial noise and achieves minimax optimality for Hölder functions with smoothness ≤1, and when combined with kernel smoothing or local polynomial regression, it extends to higher smoothness classes with minimax results.
We consider the non-parametric regression problem under Huber's $ε$-contamination model, in which an $ε$ fraction of observations are subject to arbitrary adversarial noise. We first show that a simple local binning median step can effectively remove the adversary noise and this median estimator is minimax optimal up to absolute constants over the Hölder function class with smoothness parameters smaller than or equal to 1. Furthermore, when the underlying function has higher smoothness, we show that using local binning median as pre-preprocessing step to remove the adversarial noise, then we can apply any non-parametric estimator on top of the medians. In particular we show local median binning followed by kernel smoothing and local polynomial regression achieve minimaxity over Hölder and Sobolev classes with arbitrary smoothness parameters. Our main proof technique is a decoupled analysis of adversary noise and stochastic noise, which can be potentially applied to other robust estimation problems. We also provide numerical results to verify the effectiveness of our proposed methods.