Alternating Estimation for Structured High-Dimensional Multi-Response Models
This work addresses the challenge of parameter estimation in high-dimensional multi-response models, which is incremental as it extends existing methods with new theoretical guarantees.
The paper tackles the problem of learning high-dimensional multi-response linear models with structured parameters by proposing an alternating estimation (AltEst) procedure that exploits noise correlations among responses. It shows that the error of the estimates converges linearly to a minimum achievable level under certain assumptions, providing the first non-asymptotic statistical guarantee for such algorithms with general structures.
We consider learning high-dimensional multi-response linear models with structured parameters. By exploiting the noise correlations among responses, we propose an alternating estimation (AltEst) procedure to estimate the model parameters based on the generalized Dantzig selector. Under suitable sample size and resampling assumptions, we show that the error of the estimates generated by AltEst, with high probability, converges linearly to certain minimum achievable level, which can be tersely expressed by a few geometric measures, such as Gaussian width of sets related to the parameter structure. To the best of our knowledge, this is the first non-asymptotic statistical guarantee for such AltEst-type algorithm applied to estimation problem with general structures.