Learning with Square Loss: Localization through Offset Rademacher Complexity
This provides theoretical guarantees for regression without boundedness assumptions, extending prior bounded-case results.
The paper tackles regression with square loss for unbounded function classes by introducing offset Rademacher complexity to analyze localization, showing that a two-step estimator's excess loss is bounded by this complexity through a geometric inequality.
We consider regression with square loss and general classes of functions without the boundedness assumption. We introduce a notion of offset Rademacher complexity that provides a transparent way to study localization both in expectation and in high probability. For any (possibly non-convex) class, the excess loss of a two-step estimator is shown to be upper bounded by this offset complexity through a novel geometric inequality. In the convex case, the estimator reduces to an empirical risk minimizer. The method recovers the results of \citep{RakSriTsy15} for the bounded case while also providing guarantees without the boundedness assumption.