Generic Error Bounds for the Generalized Lasso with Sub-Exponential Data
This work extends theoretical guarantees for the generalized Lasso beyond Gaussian assumptions, addressing high-dimensional estimation problems in statistics and machine learning, but it is incremental as it builds on prior research.
The paper tackles the problem of non-asymptotic error bounds for the generalized Lasso with sub-exponential data, showing that estimation error can be controlled using two complexity parameters derived from a generic chaining proof, and applies this to semi-parametric models and phase retrieval.
This work performs a non-asymptotic analysis of the generalized Lasso under the assumption of sub-exponential data. Our main results continue recent research on the benchmark case of (sub-)Gaussian sample distributions and thereby explore what conclusions are still valid when going beyond. While many statistical features remain unaffected (e.g., consistency and error decay rates), the key difference becomes manifested in how the complexity of the hypothesis set is measured. It turns out that the estimation error can be controlled by means of two complexity parameters that arise naturally from a generic-chaining-based proof strategy. The output model can be non-realizable, while the only requirement for the input vector is a generic concentration inequality of Bernstein-type, which can be implemented for a variety of sub-exponential distributions. This abstract approach allows us to reproduce, unify, and extend previously known guarantees for the generalized Lasso. In particular, we present applications to semi-parametric output models and phase retrieval via the lifted Lasso. Moreover, our findings are discussed in the context of sparse recovery and high-dimensional estimation problems.