Learning Heteroscedastic Models by Convex Programming under Group Sparsity
This addresses a major obstacle in high-dimensional regression for domains like time series where noise variance is hard to know, though it appears incremental as an extension of existing sparse methods to heteroscedastic cases.
The paper tackles the problem of applying sparse estimation methods like Lasso in settings with unknown or heteroscedastic noise, such as time series or inverse problems, by proposing a joint estimator for conditional mean and variance that is computable via second-order cone programming, with theoretical and numerical results showing its effectiveness.
Popular sparse estimation methods based on $\ell_1$-relaxation, such as the Lasso and the Dantzig selector, require the knowledge of the variance of the noise in order to properly tune the regularization parameter. This constitutes a major obstacle in applying these methods in several frameworks---such as time series, random fields, inverse problems---for which the noise is rarely homoscedastic and its level is hard to know in advance. In this paper, we propose a new approach to the joint estimation of the conditional mean and the conditional variance in a high-dimensional (auto-) regression setting. An attractive feature of the proposed estimator is that it is efficiently computable even for very large scale problems by solving a second-order cone program (SOCP). We present theoretical analysis and numerical results assessing the performance of the proposed procedure.