On the Prediction Performance of the Lasso
This provides insights into Lasso's limitations for practitioners in statistics and machine learning, though it is incremental as it builds on existing Lasso theory.
The paper investigates how the prediction performance of the Lasso relates to covariate correlations in multiple linear regression, showing that incorporating a correlation measure into the tuning parameter yields nearly optimal performance for highly correlated covariates, but performance remains mediocre for moderate correlations regardless of tuning.
Although the Lasso has been extensively studied, the relationship between its prediction performance and the correlations of the covariates is not fully understood. In this paper, we give new insights into this relationship in the context of multiple linear regression. We show, in particular, that the incorporation of a simple correlation measure into the tuning parameter can lead to a nearly optimal prediction performance of the Lasso even for highly correlated covariates. However, we also reveal that for moderately correlated covariates, the prediction performance of the Lasso can be mediocre irrespective of the choice of the tuning parameter. We finally show that our results also lead to near-optimal rates for the least-squares estimator with total variation penalty.