A concentration inequality for the excess risk in least-squares regression with random design and heteroscedastic noise
This provides a theoretical foundation for analyzing regression models with complex noise structures, though it is incremental as it builds on existing methods for linear contrast.
The paper tackles the problem of deriving a concentration inequality for the excess risk in least-squares regression with random design and heteroscedastic noise, achieving a general result that extends previous work from linear to quadratic contrast estimation.
We prove a new and general concentration inequality for the excess risk in least-squares regression with random design and heteroscedastic noise. No specific structure is required on the model, except the existence of a suitable function that controls the local suprema of the empirical process. So far, only the case of linear contrast estimation was tackled in the literature with this level of generality on the model. We solve here the case of a quadratic contrast, by separating the behavior of a linearized empirical process and the empirical process driven by the squares of functions of models.