Non-negative least squares for high-dimensional linear models: consistency and sparse recovery without regularization
This work challenges the necessity of sparsity-promoting regularization in high-dimensional settings, offering a simpler, parameter-free alternative for applications where non-negativity constraints are natural.
The paper tackles the problem of high-dimensional linear regression where regularization is typically required, showing that non-negative least squares (NNLS) can achieve comparable prediction and estimation performance to the lasso under certain design conditions, with specific advantages in support recovery when combined with thresholding.
Least squares fitting is in general not useful for high-dimensional linear models, in which the number of predictors is of the same or even larger order of magnitude than the number of samples. Theory developed in recent years has coined a paradigm according to which sparsity-promoting regularization is regarded as a necessity in such setting. Deviating from this paradigm, we show that non-negativity constraints on the regression coefficients may be similarly effective as explicit regularization if the design matrix has additional properties, which are met in several applications of non-negative least squares (NNLS). We show that for these designs, the performance of NNLS with regard to prediction and estimation is comparable to that of the lasso. We argue further that in specific cases, NNLS may have a better $\ell_{\infty}$-rate in estimation and hence also advantages with respect to support recovery when combined with thresholding. From a practical point of view, NNLS does not depend on a regularization parameter and is hence easier to use.