Support recovery and sup-norm convergence rates for sparse pivotal estimation
This work addresses theoretical guarantees for sparse regression methods, which is important for statisticians and machine learning practitioners, though it appears incremental as it extends existing pivotal estimators with smoothing techniques.
The paper tackles the problem of establishing sup-norm convergence rates for sparse pivotal estimators like the square-root Lasso in high-dimensional regression, showing minimax rates for both non-smoothed and smoothed versions, and provides guidelines for setting the smoothing hyperparameter with synthetic data validation.
In high dimensional sparse regression, pivotal estimators are estimators for which the optimal regularization parameter is independent of the noise level. The canonical pivotal estimator is the square-root Lasso, formulated along with its derivatives as a "non-smooth + non-smooth" optimization problem. Modern techniques to solve these include smoothing the datafitting term, to benefit from fast efficient proximal algorithms. In this work we show minimax sup-norm convergence rates for non smoothed and smoothed, single task and multitask square-root Lasso-type estimators. Thanks to our theoretical analysis, we provide some guidelines on how to set the smoothing hyperparameter, and illustrate on synthetic data the interest of such guidelines.