Model selection with lasso-zero: adding straw to the haystack to better find needles
This addresses the challenge of accurate model selection in high-dimensional statistics, offering a novel method that improves upon existing techniques like Lasso, though it is incremental in nature.
The paper tackles the problem of recovering the support of a sparse vector in high-dimensional linear models by introducing Lasso-Zero, which uses an 'overfit, then threshold' paradigm with noise dictionaries to improve support recovery, achieving a high true positive rate and low false discovery rate compared to competitors.
The high-dimensional linear model $y = X β^0 + ε$ is considered and the focus is put on the problem of recovering the support $S^0$ of the sparse vector $β^0.$ We introduce Lasso-Zero, a new $\ell_1$-based estimator whose novelty resides in an "overfit, then threshold" paradigm and the use of noise dictionaries concatenated to $X$ for overfitting the response. To select the threshold, we employ the quantile universal threshold based on a pivotal statistic that requires neither knowledge nor preliminary estimation of the noise level. Numerical simulations show that Lasso-Zero performs well in terms of support recovery and provides an excellent trade-off between high true positive rate and low false discovery rate compared to competitors. Our methodology is supported by theoretical results showing that when no noise dictionary is used, Lasso-Zero recovers the signs of $β^0$ under weaker conditions on $X$ and $S^0$ than the Lasso and achieves sign consistency for correlated Gaussian designs. The use of noise dictionary improves the procedure for low signals.