On the Differences between L2-Boosting and the Lasso
This addresses a theoretical gap for researchers in high-dimensional statistics and machine learning, clarifying limitations of L2-Boosting compared to l1-penalized methods, but it is incremental as it builds on known properties.
The paper tackles the problem of parameter recovery in high-dimensional linear models by proving that L2-Boosting lacks a theoretical property (restricted nullspace property) that guarantees sparse recovery in l1-penalized methods like the Lasso, showing it behaves differently in this context.
We prove that L2-Boosting lacks a theoretical property which is central to the behaviour of l1-penalized methods such as basis pursuit and the Lasso: Whereas l1-penalized methods are guaranteed to recover the sparse parameter vector in a high-dimensional linear model under an appropriate restricted nullspace property, L2-Boosting is not guaranteed to do so. Hence, L2-Boosting behaves quite differently from l1-penalized methods when it comes to parameter recovery/estimation in high-dimensional linear models.